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[57] ABSTRACT 

A post -processor for a decoded video sequence includes a 
digital noise reduction unit and an artifact reduction unit 
which significantly reduce blocking artifacts and mosquito 
noise in a video image. The post -processor uses both tem- 
poral and edge characteristics of the video image to enhance 
the displayed image. A coding parameter from a decoder is 
used in a coding parameter adaptive filter unit within an 
artifact unit to further enhance the perceived quality of the 
displayed image. The coding parameter for a particular 
macroblock is selected using a characteristic of that mac- 
roblock. The post-processor operates on a current frame of 
pixel data using information from the immediately preced- 
ing post-processed frame that is stored in a frame memory 
of the post-processor. The post-processor uses artifact reduc- 
tion only on portions of the image that are not part of an 
edge, and are not part of a texture or fine detail area. Since 
artifact reduction is not utilized on these areas, the post- 
processed image is not softened in regions where it is easily 
noticed by the human eye. 

40 Claims, 24 Drawing Sheets 
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CODING PARAMETER ADAPTIVE ously utilized in post-processing to address the shortcomings 

TRANSFORM ARTIFACT REDUCTION of spatially-invariant filtering. Sec, for example, "Nonlinear 

PROCESS space-variant postprocessing of block coded images," by 

Ramamurthi and Gersho in IEEE transactions on Acoustics, 

CROSS REFERENCE TO RELATED 5 Speech, Signal Processing, vol. ASSP-34, pages 1258-1264, 

APPLICATION October 1986, and "Contour based Post Processing of 

This application is a continuation-in-part of copending ^Images" by Y S ' "° "* ^7 SPIE 

and commonly assigned U.S. patent application Ser. No jk 119 """^^ ^1 ^u^^Jr^' 

08/486,279, entitled "Transform Reduction Process" of S. November 1989,pages 1440-1449 and "Edge Based Post 

Gupta and Y. T. Tse filed on Jun. 6, 1995. 10 Passing by WJham E. Lynch, Ph.D. thesis, Dept. of 

Electrical Engineering, Princeton University, 1993, pp. 

FIELD OF THE INVENTION 51-76. However, such filtering processes require sophisti- 
cated processors and have significant memory requirements. 

This invention relates generally to post processing of a The complexity of this approach severely limits the appli- 

decoded digital image and in particular to post-processing i 5 cations for which spatially-variant filtering, is practical. In 

that significantly reduces blocking artifacts and mosquito particular, these solutions are not well-suited for either 

noise without compromising the overall perceived quality of real-time applications or moderately priced systems, 

the image. Additionally, these post-processors are for still images and 

INSCRIPTION of RFI ATFn ART do not 1156 tem P oral characteristics. These post-processors 

DESCRIPTION OF RELATED ART 2Q &]&Q dQ Qo{ usc paramcters to dean up tfac artifacts . 

Low bit rate video encoding introduces visually objec- Nevertheless, post -processing is desirable. Since post- 
tionable quantization artifacts in reconstructed images. The processing is performed on the decoded image, no modifi- 
perceived quality of reconstructed images can be improved cations are made to the encoded bit-stream transmitted over 
by post -processing of the decoded data. In a video commu- communication channel 103. Therefore a video receiver 
nication system 100 without post-processing, a video image 25 with post-processor 205 can provide better performance than 
signal 101 drives a video encoder 102. Video encoder 102 decoder 104 without a post-processor (shown in FIG. 1) 
typically encodes a frame in video signal by processing while remaining compatible with existing encoders. Post- 
blocks of pixels within the frame. Video encoder 102 processing can be applied to any system including those 
typically divides each frame into non-overlapping blocks of which are standard compliant to gain a competitive advan- 
data and then transforms these blocks of data. In most 30 tage over other nominally standard compliant systems, 
applications, a discrete cosine transform is used. The trans- However, a post-processing method is needed that requires 
formed data is quantized, and the quantized data is driven neither excessive processing performance nor memory 
onto a communication channel 103. storage, but provides an enhanced picture quality without 

Decoder 104 receives the quantized data from communi- blurring of edges and edge boundary regions, 

cation channel 103 and performs operations that are the 35 

inverse of those performed by encoder 102 to reconstruct a SUMMARY OF THE INVENTION 

video image that represents original video image signal 101 According to the principles of this invention, a post- 

for display on display unit 106. In particular, an inverse proC essor for a decoded video sequence includes a digital 

discrete cosine transform is used if encoder 102 employed a noise reduction ^ and an artifact reduction unit w £ ch 

discrete cosine transform. significantly reduce blocking artifacts and mosquito noise in 

Discrete cosine transformation based compression, espe- a video image. Preferably, the post-processor uses both 

cially at low and moderate bit rates, results in two distinct temporal and edge characteristics of the video image to 

type of artifacts in the reconstructed video image. A first type enhance the displayed image. However, post-processing 

of artifacts is referred to as mosquito noise that is ringing 45 based only upon edge characteristics, according to the 

around sharp edges in the video image that results from principles of this invention, represents a significant improve - 

attenuation of high frequency transform coefficients. These me nt over the prior art post-processors that utilized spatially 

artifacts show up as high frequency noise patterns around invariant filtering. 

the edges of the decoded images. The post . processor of this invent ion operates on a current 

A second type of artifacts is referred to as blocking. 50 frame of pixel data using information from the immediately 

Blocking is visible tiling in a smooth area of the recon- preceding post-processed frame that is stored in a frame 

structed video image since smooth transitions are replaced memory of the post-processor. Since the human eye can 

by abrupt changes introduced by the quantization. The size easily discern textures and fine details in stationary areas 

of tiles is the same as the block size used in the transform within an image sequence, the post-processor preserves 

coding, typically eight-by-eight pixels. 55 textures and low-level details in these stationary areas. Only 

Since both of the above artifacts are high frequency in stationary areas are of concern, because motion masking is 

nature, an intuitive way to reduce the artifacts is by low pass such that texture loss is not noticed in the moving areas, 

filtering the decoded image with a space-invariant filter in a Specifically, in one embodiment, the post-processor first 

post-processor 205 (FIG. 2) prior to providing the recon- identifies texture and fine detail areas in the decoded image, 

structed video signal to display unit 106. Such an approach 60 hereinafter, image. The post-processor uses artifact reduc- 

often referred to as "spatially invariant filtering" has the tion only on portions of the image that are not part of an 

undesirable side-effect of blurring details such as sharp edge, and are not part of a texture or fine detail area. Since 

edges. Blurring severely degrades the overall perceived artifact reduction is not utilized on these areas by the 

quality of the image. post-processor, the post-processed image is not softened in 

Clearly, a more sophisticated approach is needed to clean 65 regions where the softening is easily noticed by the human 

up the artifacts while preserving the sharpness in the eye. In another embodiment, the post-processor uses infor- 

decoded image. Spatially-variant filtering has been previ- mation contained in the coding parameters, that are used by 
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the decoder, to vary the extent of post-processing. The plurality of edge classifications used in the filtering 

Specifically, coarsely quantized areas are post-processed process includes an edge classification, and the pixel is 

more heavily than finely quantized areas. assigned to edge classification if the edge flag for the pixel 

The digital noise reduction unit in the post-processor in the edge map indicates that the pixel is an edge pixel. A 

attenuates small differences between each arriving pixel and 5 pixel assigned an edge classification is unchanged by the 

the corresponding pixel from the preceding frame in the filtering process. 

frame memory. Preferably, the digital noise reduction is classiflcation m me luralit of ed classiflca . 

recursive . 

^ , . „ . , .~ . , P tions is an edge border classification. A pixel is assigned the 

TTie artifact reduction unit first identifies a pixel as one of ed borfer classification if (i) the ed fl fof at f east one 

an edge pixel and a non-edge pixel and hen sets an edge flag 30 ^ m { d 6f . fa ^ fa ^ f ( - 

for the pixel in an edge map if the pixel was identified as an v , „ £ . ti [ ■ , • . 7 * . , K I 

edge pixel. Using the edge map information for a current e * c fla ? for at least thre f f^f in a ™ nd ™ 0 fP lxc ^ about 

pixel and the pixels in a window surrounding the current ! he P lxe ! are not s _f t; and ( m ) the P Kel 15 m a luie of P lxels 

pixel, the artifact reduction unit classifies the current pixel as m tQe wi ndow and the edge flag for each pixel in the fine is 

one of an edge pixel, an edge border pixel, and a shade pixel. not ^ A P lxel assigned the edge border classification is 

Edge pixels are not filtered by the artifact reduction unit. 35 fi f tered in a one-dimensional filter that processes the line of 

Edge border pixels are filtered with a one-dimensional filter, pixels. 

and shade pixels are filtered with a two dimensional filter. Yet another classification in the plurality of edge classi- 
Controlling this spatially-variant filtering by only infer- fications is a shade classification. A pixel is assigned the 
mation in an edge map for a window about the current pixel shade classification if the edge flag for the pixel and each 
provides a significant reduction in blocking artifacts. The pixel in a window of pixels about the pixel is not set. A pixel 
combination of digital noise reduction and the spatially- assigned the shade classification is filtered in a two- 
variant filtering of the artifact reduction unit provides an dimensional filter that processes the window of pixels, 
even better reduction in blocking artifacts and mosquito The novel method for spatially-variant filtering to reduce 
noise. 25 transform artifacts includes: 

Thus, according to the principles of this invention, a assigning a pixel in a block of pixels one classification in 

transform artifact reduction method for decoded video pixel a plurality of edge classifications using edge flags in an 

data includes performing digital noise reduction on a block edge map for pixels in a window about the pixel 

of pixels to obtain a digitally noise reduced block of pixels, wherein an edge flag for a pixel is set in the edge map 

and filtering the digitally noise reduced block of pixels using 30 to indicate the pixel is in an edge of a video image; and 

a spatially-variant filter To generate the edge map used in filtering each pixel ^ the block of ixeb baged the 

thus method, a pixel gradient for the pixel is compared with classification t0 rcducc transform artifacts in 

a threshold. Hie edge flag tor the pixel in the edge map is set tne v i deo image 

upon the pixel gradient being greater than the threshold. In In the filtering process pixels directly adjacent to ed 

one embodiment, the pixel gradient is compared with an 35 are not processed with a two-dimensional filter because such 

adaptive edge threshold. In another embodiment the pixel a fiUer would indude pixelg from eithef side of afl ed 

gradient is compared with a uminance threshold. In yet However, it is desirable to clean up the area next to edges 

another embodiment, the pixel gradient is compared with (edge bordcf areas) tQ the maximum extcrjt possible without 

both an adaptive edge threshold and a luminance threshold. smearing the ed since ^ in deaQ SQ ed 

The edge flag for the pixel is set only upon the pixel gradient 40 which m critical t0 the perceived quality of an ^ 

being greater than the adaptive edge threshold, and being # at least three pixels in the window mdudmg ^ 

greater than the luminance threshold. pixel are not edge pixds and at least Qne pixel in me current 

To generate the luminance threshold, a background lumi- window is an edge pixel, the window is examined to see if 

nance measure for the pixel is generated. The luminance all the pixels lying along onc of the four possi51e axes 

threshold is proportional to the background luminance mea- 45 through the window are not edge pixels. If an axis is made 

sure. The background luminance measure is generated by up of non-edge pixels, the pixels on that axis are processed 

averaging luminance components in a window about the w i t h a one-dimensional filter. The four possible axes are 

pixel. In one embodiment, the window is a three pixels-by- checked sequentially and the one-dimensional filtering is 

three pixels window with the pixel centered in the three performed along the first axis for which all the pixels are 

pixels-by-three pixels window. The luminance threshold is 50 n ot-edge pixels. Axis examination is stopped after the first 

defined as the maximum of a minimum luminance threshold axis along which altering is allowed is found. Although 

and an adjusted luminance measure. adjacent pixels are not examined for continuity of direction, 

The pixel gradient is generated from a plurality of spatial axis examination always proceeds in a predetermined order, 

gradients for the pixel again using pixels in the window This ensures that adjacent pixels are classified similarly if 

about the pixel. In one embodiment the plurality of spatial 55 ambiguity in classification exists. 

gradients comprises two spatial gradients and in another " This process of axis selection and resultant one- 
embodiment four spatial gradients. The plurality of spatial dimensional directional filtering is equivalent to finding 
gradients is combined to obtain the pixel gradient. pixels adjacent to the edges, i.e, finding edge border areas, 
The adaptive edge threshold is generated by first gener- and filtering pixels in the edge border areas along a direction 
ating an edge estimator for a plurality of edges through the 60 parallel to the edges. This technique also provides edge 
pixel to obtain a plurality of edge estimators. Again, pixels enhancement. 

in a window about the pixel are used to generate the plurality Edge pixels and pixels directly adjacent to the edge pixels 

of edge estimators. In one embodiment, the plurality of edge that are not selected for one-dimensional directional filtering 

estimators is two pixel texture estimators and in another are not post-processed. Leaving these pixels unchanged 

embodiment four pixel texture estimators. The edge estima- 65 ensures that sharpness of edges in the decoded image is not 

tors in the plurality are combined to obtain the adaptive edge degraded by post-processing. This implies that the pixels 

threshold. adjacent to edges which do not qualify for one -dimensional 
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directional filtering are also treated like edges. This is operations of the artifact reduction control unit and the 

equivalent to coalescing edge segments with a small dis- switched filter unit. 

continuity (1 to 2 pixels) into continuous contours. To this FIG. 5D is a continuation of the process flow diagram of 

extent, the pixel classification process in the filtering process FIGS 5A 5B and 5C 

compensates for the lack of edge linking and tracing and 5 ' ' * 

allows the post-processor to effectively use an edge map FIG " 6 1S a more detailed process diagram of the opera- 
equivalent to those generated by more complex edge detec- tl0ns 1D the cd 8 c detector of FIG. 5B. 
tors. FIG. 7 illustrates the representations used for the pixels in 

In one embodiment, only a spatially-variant filter was the 3x3 windows utilized in this invention, 

used in the artifact reduction unit. However, further 1Q FIGS. 8A to 8D illustrate the various edges that are 

enhancements in both picture quality and performance are detected in the 3x3 window according to the principles of 

obtained by taking advantage of knowledge of a coding this invention. 

parameter or -coding parameters used in i the .decoder for pixel FIG. 9 is a more detailed process flow diagram of the 

filtering in the artifact reduction unit. In this embodiment a switched fihcr unit of ^ mventioa> 

coding parameter adaptive filter is included m the artifact ~™ , 

reduction unit FIGS * 10A t0 10D lllustrates tne various conditions that 

Acoding parameter adaptive post-processor adapts to the arc checked in thc switched filter unit of this invention, 

coding parameters available at the decoder, e.g., a quantizer FIG * 11 is a Dlock diagram of a quantizer scale adaptive 

scale is used in a coding parameter adaptive filter. Further, post-processor, according to the principles of this invention, 

a characteristic or characteristics of a macroblock are used 20 FIG. 12 is a process flow diagram for one embodiment of 

in selecting a specific coding parameter for use in the coding a procedure for loading quantizer scales in the quantizer 

parameter adaptive filter. Thus, the post-processor is adapted scale adaptive post-processor of FIG. 11, according to the 

to each macroblock. principles of this invention. 

Since the quantizer scale coding parameter is indicative of FIG. 13 is a diagram of a frame memory used in the 

the degree of quantization, the quantizer scale also is indica- 25 quantizer scale adaptive post-processor of this invention 

tive of the amount of post-processing that is needed. In with definitions of the various parameters used in defining 

addition, to using the quantizer scale in the post-processing, the strip of pixels that are post-processed, 

the components in this embodiment of the quantizer scale FIG. 14A is a process flow diagram for one embodiment 

adaptive post-processor have been simplified to assxst in of the initialization procedure for the quantizer scale adap- 

real-hme implementation. 30 tive post-processor of this invention. 

If quantization errors are minimal, coding artifacts are „ . a , , „. 

also minimal, and therefore the post-processing should be /} G ' 14B 15 a P 10 ^ 55 fl ° w u dia f am . for one embodiment 

minimal. However if quantization errors are large, the cod- of lhe operations performed by the digital noise reduction 

ing artifacts are severe and the sequence should be heavily umt m . me W™*™ adaptive post -processor of this 

post-processed. One embodiment of the coding parameter 35 inventlon - 

adaptive post-processor utilizes digital noise reduction, edge FlG - 14C k a process flow diagram for one embodiment 

detection, switched filtering and a coding parameter of the operations performed by the edge pre-processor in the 

adjustment, e.g., a quantization parameter adjustment. quantizer scale adaptive post-processor of this invention. 

The artifact reduction unit of this invention was success- FIG. 14D is a process flow diagram for one embodiment 

ful in eliminating most of the coding artifacts while pre- 4Q of the operations performed by the edge detection unit in the 

serving edges. The visual improvement in the quality of low quantizer scale adaptive post-processor of this invention, 

to medium bit rate coded images was striking. Extensive FIG. 14E is a process flow diagram for one embodiment 

simulations show that the artifact reduction unit substan- of the operations performed by the switched filter unit in the 

tially and dramatically improves the performance of low to quantizer scale adaptive post-processor of this invention, 

medium bit rate video decoders bycleaning coding artifacts 45 FIG. 14F is a process flow diagram for one embodiment 

while preserving edge sharpness^The post-processor can be of ^ rations formed 5 ^ ^ scale ad 

added to any video receiver between decoder and display tion unil ^ the S J Q p0St . proces J. of 

modules to enhance the displayed image. thig ; nvenl i on 

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 14G is a process flow diagram for one embodiment 

FIG. 1 is a block diagram of a prior art video communi- 50 of the operations performed by the quantizer scale adaptive 

cation system without post -processing. post-processor of this invention upon completion of pro- 

FIG. 2 is a block diagram of a prior art video communi- cessing of a strip of data. 

cation system with post -processing. FIG. 15 is a diagram that illustrates the pixel used in 

FIG. 3 is a block diagram of a novel post-processor 55 generating the weighted three pixel horizontal and vertical 

according to the principles of this invention for use in video spatial gradient factors. 

communication systems such as those illustrated in FIG. 1. FIG. 16 is a more detailed process flow diagram for the 

FIG. 4 is a more detailed block diagram of one embodi- filter unit within the switched filter unit of the quantizer scale 

ment of the post-processor of this invention. adaptive post-processor of this invention. 

FIG. 5A is a process flow diagram of the initialization 60 FIG. 17A is a first application of the quantizer scale 

process of the post -processor of FIG. 4 and of one embodi- adaptive post-processor of this invention in conjunction with 

ment of the operations of the digital noise reduction unit. translating a frame in a first formal to a frame in a second 

FIG. SB is a continuation of the process flow diagram of format. 

FIG. 5A that illustrates one embodiment of the operations of FIG. 17B is a second application of the quantizer scale 

the edge detection unit. 65 adaptive post-processor of this invention in conjunction with 

FIG. 5C is a continuation of the process flow diagram of translating a frame in a first format to a frame in a second 

FIGS. 5A and 5B that illustrates one embodiment of the format. 
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FIGS. 18Ato 18C are diagrams that illustrate the various ences between each arriving pixel and the corresponding 

scales used by the vertical interpolation unit of FIG. 17B. pixel from preceding frame Fn_l in frame memory 301. 

FIG. 19 is a block diagram of another embodiment of the Digital noise reduction unit 310 treats small differences 

quantizer scale adaptive post-processor of this invention. between pixels in corresponding frames as noise and mini- 

5 mizes these small differences. Larger pixel differences are 
treated as signal and are not attenuated by digital noise 

According to the principles of this invention, a post- reduction unit 310. 

processor 300 overcomes the limitations of both spatially * ■ , iiL , j • i * ■ 

invariant filtering and spatially-variant filtering of prior art 10 * * known 0 those ™ m the a ?> frame FnJ ? a 

post-processors. This novel post-processor 300 cleans up two-dimensional array of pixel data in frame memory 301 

mosquito noise and reduces blocking artifacts while pre- and (n_l) is a temporal index that denotes the (n-l)th frame 

serving image sharpness. Preferably, post-processor 300 m a tuiie sequence of frames where n can take on any value 

uses both temporal and edge characteristics of a video image from one for the second frame in the time sequence to (N) 

to enhance the displayed image. However, post-processing 35 where N is the last frame in the time sequence. Pixel 

based only upon edge characteristics, as described more Fn__l(ij) is a pixel in the frame for the temporal time (n-1) 

completely later, represents a significant improvement over m the with row and jth column. In this embodiment, frame 

the prior art post-processors that utilized spatially invariant FnJ in me 301 has been processed by post-processor 

filtering. 300 

FIG. 3 is a block diagram of one embodiment of post- 20 

processor 300 of this invention. Post-processor 300 operates Similarly frame Pn is a two-dimensional array of decoded 

on a current frame of pixel data Pn using information from pixel data that is available from the decoder where again n 

the immediately preceding post-processed frame Fn„l that is a temporal index that denotes the nth frame from the 

is stored in a frame memory 301. Since the human eye can decoder in a time sequence of frames where n can take on 

easily discern textures and fine details in stationary areas 2 5 any value from 1 for the first frame in the time sequence to 

within an image sequence, post-processor 300 preserves N for the last frame in the lime sequencet Ptl(i : } ^ a 

texhires and low-level details in these stationary areas Only ^ in the frame fof ^ ^ ^ n {n fow and 

stationary areas are of concern, because motion masking is f ^ co j umn 

such that texture loss is not noticed in the moving areas. * C0 Umn ' 

Specifically, post-processor 300, as explained more com- 30 Output pixel Qn(i,j) from digital noise reduction unit 310 

pletely below, first identifies texture and fine detail areas in for input pixel Pn(ij) is: 
the decoded image, hereinafter, image. Post-processor 300 

uses artifact reduction only on portions of the image that are Qn(y>Fn_i&/)+Adn(i;;)) (1) 
not part of an edge. As described more completely below, 

post-processor 300 also uses texture information to ensure 35 wnere 
that the artifact reduction is not utilized in regions where 

image softening is easily noticed by the human eye. f ( ) is a predetermined digital noise reduction function; 

Post-processor 300 has two major components, a digital and 
noise reduction unit 310 and an artifact reduction unit 350. 

As explained more completely below, digital noise reduction 40 <Nk>(Pn(y)-Fa_i(y)) (2) 
unit 310 attenuates small differences between each arriving 

pixel and the corresponding pixel from preceding frame The value of di 8 ital noise reduction function f(dn(i j)) is 

Fn_l in frame memory 301. obtained from a stored look-up table, in this embodiment, 

Artifact reduction unit 350 first identifies a pixel as one of Dased on di gi tal P^ 1 difference dn(i j). Since digital noise 

an edge pixel and a non-edge pixel and sets an edge flag for 45 reduction unit 310 uses frame Fn_l(i j) from post-processor 

the pixel in an edge map if the pixel is identified as an edge 300 rather than the previous frame Pn„l(i j) from the 

pixel. The edge map is a plurality of edge flags where for decoder, the digital noise reduction is recursive, 

each pixel, the edge flag is set if the pixel is on an edge and 0ne embodiment of a look-up table suitable for use with 
is cleared otherwise 

50 this invention is given in Table 1. If the absolute value of 

Using the edge map information for a current pixel and di hjd ixd difference dn(ij) is less that sixte di ^ tal 

the pixels in a window surrounding the current pixel, artifact rq[s& reduced M * 

reduction unit 350 classifies the current pixel as one of an D r „ ; K ,, v , v . . — . v 7 " 

edge pixel, an edge boundary pixel, and a shade pixel. Edge lf a 1 bsolutc val » c of dl & ltal P™ 1 

pixels are not filtered by artifact reduction unit 350. Edge „ dlfferenc / » greater than or equal to sixteen, digital 

boundary pixels are filtered with a one-dimensional filter, noise reduced 0Ut P ut P lxel Qn ^ 15 ^ same as in P ul P™ [ 

and shade pixels are filtered with a two-dimensional filter. ?n ^' where the difference between current input 

Performing this spatially-variant filtering using only P lxeI Pn ^ and the ^"^ponding P*el in the previous 

information in an edge map for a window about the current P°st-processed frame, i.e, pixel Fn_l(i j) is between -16 

pixel provides a significant reduction in blocking artifacts 60 and +16 » current 1D P ut P lxel Pn ( J j) fa modified so that 

and mosquito noise, as described more completely below. frame-to-frame pixel differences are reduced. 

The combination of digital noise reduction and the spatially- Table 1 gives the values of digital noise reduction function 

variant filtering of artifact reduction unit 350 provides an f(dn) for positive digital pixel differences dn(ij). Corre- 

even better reduction in blocking artifacts and mosquito sponding values of digital noise reduction function f(dn) for 

noise. 65 negative digital pixel differences dn(ij) are obtained by 

Thus, in this embodiment, post-processor 300 first uses a placing negative signs on digital pixel difference dn and 

digital noise reduction unit 310 to attenuate small differ- digital noise reduction function f(dn) in Table 1. 
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greater than a texture pixel threshold, and the pixel is not an 
TABLE 1 edge pixel, a texture pixel counter is incremented for the 

block, i.e, the pixel is identified as a texture pixel. 

After every pixel in the block is processed in edge 
5 detection unit 320, edge detection unit 320 sets an edge 
block flag Eb for a block when the block has more edge 
pixels than an edge block threshold. The state of edge block 
flag Eb distinguishes blocks with edges from blocks without 
edges. Since blocks with edges have substantial mosquito 
io noise, blocks with the edge block flag set are always 
post-processed, even though these blocks may have fine 
details or textures. 

Also after each pixel in the block is processed in edge 
detection unit 320, if the value of texture pixel counter is 
1 5 greater than a texture block threshold, a block texture flag Tb 
is set for the block. 

Thus, after a block is processed by edge detection unit 
320, a replenish flag Rb, a block texture flag Tb, and a edge 
block flag Eb have either been set or left cleared, and each 

20 pixel has been identified as an edge pixel or a not edge pixel. 

The three flags are provided to an artifact reduction control 
Digital noise reduction unit 310 also identifies unchanged unit 315 in artifact reduction unit 350. 
(stationary) areas in the image in this embodiment. The If both the replenish and texture block flags are set and the 
unchanged areas are replenished blocks which are generated edge block flag is not set, artifact reduction control unit 315 
simply by copying the block at the same location from the 25 connects digital noise reduction unit 310 to frame memory 
previous frame in the decoder. Thus, in this embodiment, 301 and to post-processor 300 output bus 302 to an output 
post-processor 300 sub -divides each frame into blocks and buffer. Thus, the digitally noise reduced block is copied to 
digital noise reduction unit 310 classifies each block indi- the output buffer and frame memory 301 without modifica- 
vidually as stationary or non-stationary, i.e., as a replenished tion. 

or non-replenished block. 30 In all other cases, artifact reduction control unit 350 

In this embodiment of digital noise reduction unit 310, an connects the output bus of switched filter unit 330 to frame 
absolute value of digital pixel difference dn(i,j) is tested memory 301 and to post-processor output bus 302. Thus, the 
against a replenished threshold. If the absolute value of block copied to frame memory 301 has had both artifact 
digital pixel difference dn(ij) is less than the replenished reduction, that is described more completely below, and 
threshold for all pixels in the block, a replenished block flag 35 digital noise reduction. 

Rb is set for the block and otherwise, replenished block flag Switched filter unit 330 in artifact reduction unit 350 uses 
Rb is cleared. Thus, in this embodiment, replenished blocks the edge map generated by edge detection unit 320 to 
are identified by examining pixel differences between cur- determine the filtering applied to each pixel in each block, 
rent frame Pn from the decoder and previous frame Fni_l Specifically, as explained more completely below, a decision 
from post-processor 300. 40 is made on the processing applied to each pixel in the block 

After digital noise reduction unit 310 processes a block by examining the edge map of pixels within a three pixels- 
and the block is flagged as replenished or non-replenished, by-three pixels (3x3) window surrounding the current pixel, 
the block is available for further processing. Initially, in the The use of edge information in a 3x3 window requires edge 
further processing, the noise reduced block is processed in information for the pixels that form a one pixel border 
an edge detection unit 320 within artifact reduction unit 350. 45 around the current window. Thus, edge detection unit 320 
As indicated above, each pixel in the noise reduced block is must provide edge information for both the current block 
classified either as an edge pixel or a non-edge pixel, i.e., an and a one pixel boundary surrounding the current block, 
edge map is generated, by edge detection unit 320. Consequently, overlapping blocks are used by post- 
Specifically, edge detection unit 320 determines whether a processor 300, as explained more completely below, 
pixel is on an edge in a block of frame Qn. Herein, an edge 50 Switched filter unit 330 performs the pixel classification 
refers to a characteristic of the video image represented by described above. Specifically, switched filter unit 330 clas- 
the frame and not a physical edge of the frame. sifies a pixel as either from an edge, edge boundary, or 

As explained more completely below, in one embodiment, "shade" (smooth) area. Pixels along edges are left 
edge detection unit 320 first compares a pixel gradient for unchanged by unit 330. Pixels along edge boundaries are 
the current pixel with a threshold, preferably an adaptive 55 directionally filtered with a one-dimensional filter. Shade 
edge threshold. An edge flag in the edge map for the current pixels are low pass filtered with a two-dimensional filter by 
pixel is set if the pixel gradient is greater than the threshold. unit 330. Thus, pixels are selectively filtered depending on 

In the process of generating the pixel gradient, a set of their classification. To prevent edge smearing, the filtering 
edge estimators are generated. In this embodiment, the edge ensures that pixels from different sides of an edge are not 
estimators include pixel texture estimators Tl(iJ) and T2(i, 60 mixed. Thus, artifact reduction unit 350 utilizes "spatially- 
j). Specifically, as every block is processed by edge detection variant switched filtering". 

unit 320, pairwise pixel texture estimators Tl(ij) and T2(i j) FIG. 4 is a more detailed block diagram of one embodi- 
along a first axis and a second axis, respectively, are gen- ment of post-processor 300 that combines digital noise 
era ted for the pixel. Here, the second axis is perpendicular reduction and artifact reduction with block classification, 
to the first axis. In this embodiment, the pairwise pixel 65 FIGS. 5 A to 5D are process flow diagrams of one embodi- 
texture estimators Tl(ij) and T2(ij) for a pixel are com- ment of the operations performed by the various units in 
bined to form a texture estimator. If the texture estimator is FIG. 4. 
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When post-processor 300 starts to process a sequence of process 505 and otherwise to initialize edge map 506. If the 

frames in a sequence, the zeroth frame is written to an input current block lies along a frame boundary, some of the pixel 

frame memory 410. Initialize frame memory process 501 locations within in-block memory 420 contain arbitrary 

detects that a new sequence is starting and copies the zeroth values. Thus, replicate pixels process 505 copies the pixels 

frame from input frame memory 410 to output frame 5 along the frame boundary into the two rows, two columns, 

memory 440 and to frame memory 301. pus, the zeroth or both of pixel locations within in-block memory 420 that 

frame is not processed by digital noise reduction unit 310 are outside of the frame boundary . mns[ pixel of the 

and artifact reduction unit 350. ....... . current block is used to fill in the pixel border in the corner 

After the zeroth frame is processed in initialize frame _ • _ Af , „ . t . t r . , 4 

memory process 501, post-processor 300 transfers to initial- n regl ° nS * ^ th * ap ? °P n * te P;^k are rephcated rephcate 

ize frame process 502. Each subsequent frame in the 30 plXel P r ^ SS 505 also transfers t0 mitialize ed S e ma P 

sequence is processed by post-processor 300 as described process 506. 

oe l ow In initialize edge map process 506, each location in 

In this embodiment, a block size of sixteen pixels-by- edge -block memory 430 is set to a predetermined value, i.e., 

sixteen pixels is used since standard codecs (H.261, MPEG) edge-block memory 430 is initialized. Specifically, if the 

use this block size for motion compensation and replenish- 35 P™* m in-block memory 420 corresponding to the location 

ment decisions, and is referred to as the current block. m edge-block memory 430 is a replicated pixel, the location 

However, those of skill in the art can select another block in edge-block memory 430 is set to indicate an edge pixel, 

size to optimize the performance for a particular application. i-e, an edge flag is set, and otherwise the location in 

Thus, the use of a block size of sixteen pixels-by-sixteen edge-block memory 430 is set to indicate not an edge pixel, 

pixels is illustrative only and is not intended to limit the 20 Also, in this embodiment, a replenishment flag Rb is set, i.e, 

invention to this particular size. Thus, for this embodiment, replenishment flag Rb is set to a first predetermined state, 

initialize frame process 502 divides the frame in input frame The size of edge-block memory is determined by the 

memory 410 into sixteen pixels-by-sixteen pixels blocks for current block size and the number of pixels required to 

subsequent processing and transfers to load block process classify each pixel in switched filter 330. Since, in this 

503. 25 embodiment, the current block size is sixteen pixels-by- 

Load block process 503 copies the current sixteen pixels- sixteen pixels, and edge information in a three pixel-by- 

by-sixteen pixels block from input frame memory 410 to three pixel window about the current pixel is required, 

in-block memory 420. The size of in-block memory 420 is edge-block memory 430 is eighteen pixels-by-eighteen pix- 

determined by the block size selected for processing and the els. 

requirements of edge detection unit 320 and switched filter 30 Upon completion of initialize edge map 506 all the 

unit 330, necessary initialization for processing of a block is 

As explained above, switched filter unit 330 requires edge complete, and so digital noise reduction unit 310 initiates 

information in a one pixel border about the current block and processing of the information in in-block memory 420, in 

so edge information is required for an eighteen pixels-by- this embodiment. Of course, if digital noise reduction is 

eighteen pixels block. As explained more completely below, 35 either unnecessary or unwanted, digital noise reduction unit 

edge detection unit 320 uses a three pixels-by-three pixels 310 could be eliminated. In this case, artifact reduction unit 

window about the current pixel to determine whether the 350 would initiate processing of the information in in-block 

current pixel is an edge pixel. Consequently, in this memory 420, as described more completely below, 

embodiment, a two pixel border is required about the current One embodiment of the process performed by digital 

block and so load block process 503 copies a twenty -by- 40 noise reduction unit 310 is illustrated in FIG. 5A. Generate 

twenty pixel block within which the current block is cen- pixel difference process 510 generates digital pixel differ- 

tered from input frame memory 410 to in-block memory 420 ence dn(i j), as defined above, using the current pixel from 

for all blocks that are not along the frame boundary. in-block memory 420 and the corresponding pixel from 

If the current block lies along the frame boundary, load frame memory 301. Upon generation of digital pixel differ- 

block process 503 can not copy a full twenty-by -twenty 45 ence dn(i j) processing transfers to access look-up table 511. 

pixel block to in-block memory 420 within which the In this embodiment, Table 1 as defined above is stored in 

current block is centered. Rather, a different size block is a memory of post-processor 300. Thus, in process 511, 

copied that contains the current block and then pixels in digital pixel difference dn(ij) is used as an index to access 

in-block memory 420 for which no pixel data is available are the appropriate value of digital noise reduction function 

initialized either by using pixel information in the copied 50 fj(dn) in the look-up table memory. Upon completion of 

block, or by setting the pixel locations to a known value, for access look-up table 511, processing transfers to output 

example zero. Each of the situations in which a full twenty- DNR pixel process 512. 

by-twenty pixel block can not be copied to in-block memory In output DNR pixel process 512, output pixel Qn(i j) is 

420 are described below. generated according to expression (1) above and loaded in 

When the current sixteen pixels-by-sixteen pixels block is 55 the appropriate location of in-block memory 420. Thus, the 

located in a corner of the frame, only an eighteen pixels- pixel in in-block memory 420 is replaced by a noise reduced 

by-eighteen pixels block is copied. A twenty pixel wide-by- pixel and processing transfers to digital pixel difference 

eighteen pixel high block is copied if the current block is not check 513. 

a corner block and is along either the top or bottom edge of In digital pixel difference check 513, an absolute value of 
the frame. An eighteen pixel wide-by-twenty pixel high 60 digital pixel difference dn(ij) for the current pixel is corn- 
block is copied if the current block is not a corner block and pared with a replenished threshold REPLTHRSH. If the 
is along either side edge of the frame. In each of these cases, absolute value of digital pixel difference dn(ij) is greater 
the copied block is appropriately located within in-block than replenished threshold REPLTHRSH, the current pixel 
memory 420. Upon completion of load block process 503, is assumed to be a new pixel and processing transfers to 
processing transfers to frame boundary check 504. 65 clear replenishment flag process 514, which in turn changes 
If the current block lies along an edge of the frame, frame the state of replenishment flag Rb to cleared, i.e., to a second 
boundary check 504 transfers processing to replicate pixels state, and then transfers to last pixel check 515. 



04/07/2004, EAST Version: 1.4.1 



5,920,356 

13 14 

Conversely, if the absolute value of digital pixel differ- In this embodiment, the pixel information in the three 

ence dn(i j) is less than replenished threshold REPLTHRSH, pixels-by-three pixels window about the current pixel is also 

processing transfers directly from check 513 to last pixel used to generate a luminance threshold. Thus, a pixel is 

check 515. In this embodiment, replenishment threshold identified as an edge pixel if the pixel gradient for the pixel 

REPLTHRSH is taken as five. Recall that frame memory 5 is greater than both the adaptive edge threshold and the 

301 contains post -processed pixels rather than pixels from luminance threshold. 

the original previous frame. Since the post-processed pixels Thus, if an edge goes through the current pixel, the current 

are different from the pixels in input frame memory 410, pixel is identified as an edge pixel by edge detector process 

replenishment threshold REPLTHRSH is preferably taken as 520 and otherwise the current pixel is identified as a not- 

a small positive threshold rather than zero. Alternatively, the 10 edge pixel. Upon completion of processing of a pixel by 

replenishment processing in DNR unit 310 could be elimi- edge detector process 520, processing transfers to edge pixel 

nated and other techniques used by post-processor 300 to check 521. If the current pixel is an edge pixel, check 521 

identify replenished blocks. transfers processing to set edge flag 522 and otherwise to 

Last pixel check 515 determines whether each pixel texture pixel check 524. 
within the current sixteen pixels-by-six teen pixels block in 15 Set edge flag 522 sets the edge flag for the current pixel 

in-block memory 420 has been processed by digital noise in edge-block memory 430 and transfers processing to 

reduction unit 310. If an additional pixel or pixels remain to update edge counter 523. Update edge counter 523 changes 

be processed, processes 510 to 515 are repeated until all the value of the edge counter, e.g., increments the edge 

pixels within the current block in memory 420 are pro- counter, to indicate the number of edge pixels in the current 

cessed. When all these pixels in memory 420 have been 20 block including the current pixel. Update edge counter 523 

processed, the current block of pixels has been replaced with also transfers processing to last pixel check 526, that is 

digitally noise reduced pixels. Also, if the absolute value of described below. 

the digital pixel difference between every pixel in the current If the current pixel is not an edge pixel, edge pixel check 

block and the corresponding pixel in reference frame 524 transfers processing to texture pixel check 524. If the 

memory 301 is less than replenishment threshold 25 sum of the pairwise pixel texture estimators Tl(ij) and 

REPLTHRSH, replenishment flag Rb is set, and otherwise T2(i,j) for the current pixel is greater than a texture pixel 

replenishment flag Rb is cleared. threshold VARTHRSH, the pixel is classified as a texture 

One embodiment of the process performed by edge detec- pixel, and so texture pixel check transfers processing to 

tion unit 320 is illustrated in FIG. 5B. In this embodiment, update texture counter 525 and otherwise to last pixel check 

upon entry to edge detection unit 320, an edge counter and 30 . 526. 

a texture counter are initialized, and a texture block flag Tb In this embodiment, texture pixel threshold VARTHRSH 

and an edge block flag Eb are both cleared, i.e., set to a first is a function of the location of the current pixel with the 

predetermined state. Each pixel within an eighteen-by- current block. Typically, an eight -by-eight pixel block size is 

eighteen pixel window centered within in-block memory used for transform coding. Thus, within the current block, 

420 is processed by edge detection unit 320. 35 there may be blocking artifacts along the edges of the 

Initially, the current pixel within the eighteen pixels-by- eight -by-eight pixel block. To prevent these blocking arti- 

eighteen pixels window is processed by edge detector pro- facts from contributing to texture pixels, texture pixel 

cess 520, as described more completely below, unless the threshold VARTHRSH is set to five for pixels not on 

edge flag for the pixel in edge-block memory 430 is already eight-by-eight pixel block borders, and is set to eight for 

set. If the edge flag is already set, a pointer to the current 40 pixels on eight-by-eight pixel block borders, 
pixel is incremented, and edge detector process 520 is In update texture counter 525, the value of the texture 

initiated again. Thus, edge detector process 520 as well as counter is changed, e.g., incremented, to indicate the number 

processes 521 to 526 are performed only for pixels that do of texture pixels in the current block including the current 

not have the edge flag already set, i.e, only for non- pixel that are not edge pixels. Update texture counter 525 

replicated pixels. 45 transfers processing to last pixel check 526. 

Edge detector process 520 uses a three pixels-by-three Last pixel check 526 determines whether each pixel in the 

pixels window about the current pixel, i.e., a total of nine eighteen-by-eighteen pixel block centered in in-block 

pixels, to determine whether there is an edge of the image memory 420 has been processed by edge detection unit 320. 

that goes through the current pixel. A plurality of spatial If an additional pixel or pixels remain to be processed, 

gradients is generated for the current pixel. The spatial 50 processes 520 to 526 are repeated, as appropriate, until all 

gradients in the plurality are combined to generate a pixel pixels within the eighteen-by-eighteen pixel block are pro- 

gradient. cessed. When all pixels in the block have been processed, 

The pixel gradient is compared to a threshold, that pref- last pixel check 526 transfers to texture block check 527. 
erably includes an adaptive edge threshold, to identify the Upon entering texture block check 527, the edge map in 

pixel as either an edge pixel or a not-edge pixel. If the pixel 55 edge-block memory 430 is updated for the current block and 

gradient is greater than the threshold, the pixel is identified the edge and texture counters reflect the number of edge 

as an edge pixel. pixels and texture pixels, respectively within the current 

As explained more completely below, to generate the block. If the value of the texture counter is greater than a 
adaptive edge threshold, the pixel information in the win- block texture pixels threshold NUM VARTHRSH, check 527 
dow about the current pixel is used to generate an edge 60 transfers processing to set block texture flag 528, which in 
estimator for each of a plurality of possible edges through turn sets the block texture flag Tb, and otherwise to edge 
the current pixel. In one embodiment, the edge estimators block check 529. In this embodiment, block texture pixels 
are pairwise pixel texture estimators Tl(ij) and T2(i j) for threshold NUMVARTHRSH is taken as sixteen. Herein, 
the current pixel along a first axis and a second axis, and the block texture pixels threshold NUMVARTHRSH was deter- 
adaptive edge threshold is called an adaptive texture thresh- 65 mined empirically. Experiments were performed on a set of 
old. The edge estimators in the plurality are combined to images and the threshold which provided results most con- 
form the adaptive edge threshold. sistent with human perception was selected. Upon comple- 
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tion of set texture block flag 528, processing transfers to 
edge block check 529. 

In edge block check 529, edge block check 529 deter- 
mines whether the value of the edge counter is greater than 
a block edge pixels threshold NUMEDGETHRSH. If the 
value of the edge counter is greater than block edge pixels 
threshold NUMEDGETHRSH, check 529 transfers process- 
ing to set edge block flag 530, which in turn sets edge block 
flag Eb, and otherwise to artifact control unit 315. In this 
embodiment, block edge pixel threshold NUMEDGETH- 
RSH is taken as eight. Block edge pixel threshold 
NUMEDGETHRSH also was determined empirically. 
Experiments were performed on a set of images and the 
threshold which provided results most consistent with 
human perception was selected. Upon completion of set 
edge block flag 530, processing transfers to artifact reduc- 
tion control unit 315. 

FIG. 5C illustrates one embodiment of the process per- 
formed by artifact reduction control unit 315. Initially, in 
texture block flag check 535, the state of texture block flag 
Tb is analyzed. If texture block flag Tb is true processing 
transfers from check 535 to edge block flag check 536, and 
otherwise to switched filter unit 330. 

In edge block flag check 536, the state of edge block flag 
Eb is analyzed. If edge block flag Eb is false processing 
transfers from check 536 to replenishment flag check 537, 
and otherwise to switched filter unit 330. 

In replenish flag check 537, the state of replenishment flag 
Rb is analyzed. If replenishment flag Rb is true, processing 
transfers from check 537 to copy current block 538, and 
otherwise to switched filter unit 330. 

In copy current block 538, the digitally noise reduced 
current sixteen -pixels by-sixteen pixels block centered in 
in-block memory 420 is copied to output frame memory 
440. When the copy is complete, copy current block 538 
transfers to last block check 552 (FIG. 5D). This is illus- 
trated in FIG. 4 by switch element 415, which is positioned 
to form a closed path from in-block memory 420 to output 
frame memory 440. 

FIG. 5C also illustrates one embodiment of the process in 
switched filter unit 330. Upon entry of switched filter unit 
330, switch element 415 connects switched filter unit 330 to 
in-block memory 420. Switched filter unit 330 processes 
each of the pixels in the current block pixel-by-pixel. 

In switched filter unit 330, each pixel is first classified as 
one of an edge pixel, an edge boundary pixel, and a shade 
pixel. Specifically, edge pixel check 541 determines whether 
the current pixel was identified as an edge pixel. If the 
current pixel is an edge pixel, no filtering is done and so 
check 541 transfers to copy pixel process 550. Copy pixel 
process 550 copies the current pixel to the appropriate 
location in sixteen pixels-by-sixteen pixels filter block 
memory 425 and transfers processing to last pixel check 
547. If the current pixel is not an edge pixel, check 541 
transfers to classify pixel process 542. 

In classify pixel process 542, the current pixel is pro- 
cessed to determine whether the current pixel is one of an 
edge boundary pixel and a shade pixel. As explained more 
completely below, the edge flag information in edge-block 
memory 430 for a three pixels-by-three pixels window about 
the current pixel is used by classify pixel process 542. After 
the current pixel is classified, processing transfers to a filter 
unit 549 within switched filter unit 330. 

In shade pixel check 543, if the current pixel is a shade 
pixel, processing transfers to two-dimensional filter 544, and 
otherwise to edge boundary pixel check 545. In two- 
dimensional filter 544, the filtered output pixel of a two- 
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dimensional filter is written in the appropriate location of 
sixteen pixels-by-sixteen pixels filter block memory 425, as 
described more completely below. Two-dimensional filter 
544, after outputting the filtered pixel, transfers processing 
to last pixel check 547. 

In edge boundary pixel check 545, if the current pixel is 
an edge boundary pixel, processing transfers to one- 
dimensional filter 546, and otherwise to copy pixel process 
551. In one -dimensional filter 546, an appropriate one- 
dimensional directional filter, as described more completely 
below, writes a filtered output pixel to the appropriate 
location in sixteen pixels-by-sixteen pixels filter block 
memory 425. One-dimensional filter 546, upon writing the 
filtered output pixel, transfers processing to last pixel check 
547. 

Copy pixel process 551 copies the current pixel to the 
appropriate location in sixteen pixels-by-sixteen pixels filter 
block memory 425 and transfers processing to last pixel 
check 547. Last pixel check 547 determines whether all the 
pixels in the current block have been processed by switched 
filter unit 330. If a pixel or pixels remain for processing, last 
pixel check 547 returns to edge pixel check 541 and pro- 
cesses 541 to 547 are repeated for the next pixel in the block. 

When all the pixels in the current block have been 
processed, last pixel check 547 transfers to copy filtered 
block 548. Copy filtered block 548 positions switch element 
435 so that digitally noise reduced and artifact reduced 
sixteen pixels-by-sixteen pixels block in sixteen pixels-by- 
sixteen pixels filter block memory 425 is copied to output 
frame memory 440. Upon completion of the copy, copy 
filtered block 548 opens switch element 435 and then 
transfers to last block check 552. 

Upon entry to last block check 552, the post-processing of 
the current block is complete. Thus, last block check 552 
determines whether there is an additional block in the frame 
in input frame memory 410 that remains to be post- 
processed. If there is at least one additional block in memory 
410, last block check 552 transfers to load block process 503 
(FIG. 5A) and the next block is processed as described 
above in references to FIGS. 5A to 5C. If there are no 
additional blocks in memory 410 that require processing, the 
post-processing of the current frame is complete and so 
processing transfers to last frame check 553. 

Last frame check 553 determines whether there is an 
additional frame in the current sequence of frames that 
remains to be post-processed. If there is at least one addi- 
tional frame in the sequence, last frame check 553 transfers 
to copy output frame process 554, which in turn copies the 
post-processed frame in output frame memory 440 to frame 
memory 301. Upon completion of the copy, process 554 
transfers to initialize frame process 502 (FIG. 5A) and the 
next frame is processed as described above in references to 
FIGS. 5 A to 5C. If there are no additional frames that require 
processing, the post-processing of the current sequence is 
complete and so processing transfers to done 555. 

The performance of post-processor 300 on data from 
codecs operating at 128 kbps has shown a significant reduc- 
tion in artifacts and mosquito noise. Several standard CIF 
test sequences were coded at 96 kbps. One hundred fifty 
frames from each test sequence were coded and post- 
processed. The decoded and post-processed frames were 
then interpolated to CCIR 601 resolution and displayed 
side-by-side for comparison. 

Substantial improvements were seen in all the test 
sequences. The combination of digital noise reduction and 
artifact reduction showed that mosquitoes were distinctly 
reduced around moving edges compared with use of artifact 
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reduction only. Furthermore, textured stationary areas in the 
background were not post-processed. Overall post-processor 
300 provides a substantial improvement in the quality of 
coded sequences with a perceptual effect judged as equiva- 
lent to almost doubling the data rate. Improvements over 5 
artifact reduction alone are obtained by incorporating pre- 
vious frame memory. Therefore, temporal information in the 
coded sequences is used to improve performance. 

In edge detector 520, each block is processed by a set of 
linear or nonlinear spatial operators which measure lumi- ]Q 
nance changes along different directions, i.e., a plurality of 
spatial gradients are generated, for each pixel. The spatial 
gradients in the plurality are combined to obtain a gradient. 
If the gradient of a pixel is sufficiently large, the pixel is 
classified as an edge pixel otherwise the pixel is classified as 
a not edge pixel. 35 

FIG. 6 is a process diagram that illustrates one embodi- 
ment of the process that edge detector 520 uses for each 
pixel in a block to determine whether the pixel lies on an 
edge in the image contained in the current frame. 
Specifically, each pixel, Q(i,j), where index i identifies the 20 
row and index j identifies the column, written in in-block 
memory 420 is processed. In this embodiment, indices i and 
j range from one to eighteen because, as explained above, an 
edge characterization is required for a one pixel border about 
the current sixteen pixels-by-sixteen pixels block. The pixels 25 
in a three pixels-by-three pixels window 700 about pixel 
Q(ij) are shown in FIG. 7. 

In initialization process 601, pixel Q(i,j) is selected where 
indices i and j initially point to the first pixel, that is not a 
replicated pixel within the eighteen pixels-by-eighteen pix- 
els block centered in in-block memory 420 and the three- 30 
by-three window of pixels. Specifically, in this embodiment, 
indices are set to address the appropriate location within 
in-block memory 420. Recall that for each replicated pixel, 
load block process 503 sets the edge flag in edge-block 
memory 430 and so it is not necessary for edge detector 520 35 
to process these pixels. Upon each subsequent entry to 
initialization process 601, the next pixel in raster scan order 
is selected. Upon completion of initialization process 601, 
edge detector 520 transfers processing to generate spatial 
gradients process 602. 40 

In this embodiment, differential operators are utilized 
within generate spatial gradients process 602 to obtain a 
plurality of spatial luminance gradients Gk(ij), where index 
k represents a particular edge orientation within three pixels- 
by-three pixels window 700. Specifically, these differential 45 
operators perform discrete differentiation on the pixels in 
three pixels-by-three pixels window 700 to measure the 
luminance change along different directions. 

Specifically, a plurality of differential operators Hk(ij) are 
based on Sobel differential operators. The Sobel differential 50 
operators used are given in expression (3). 
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Thus, this plurality of operators measures all possible 
combinations of edges through three pixels-by-three pixels 
window 700. Hence, in this embodiment of generate spatial 
gradients process 602, operators Hk(i,j), where k takes on 
values of one to four, sometimes referred to as masks Hk(i j), 
are applied to current window 700 centered on pixel Q(ij) 
to generate four spatial gradients Gk(i j) for current pixel 
Q(ij). After the spatial gradients are generated, processing 
transfers from process 602 to generate pixel gradient process 
603. 

In generate pixel gradient process 603, a point wise 
operator combines the spatial gradients in the plurality to 
obtain a pixel gradient A(i j) for pixel Q(ij). However, in the 
embodiment, the magnitude of pixel gradient A(ij) is of 
interest, and so either the squared sum or the absolute sum 
of the spatial gradients in the plurality can be used. Thus, the 
absolute sum of the spatial gradients in the plurality is taken 
as the point operator. Therefore, pixel gradient A(i,j) is: 

A(w>'/ t s(]GlMHG2(y)| + |G3C^ | + |G4ft;0|) (4) 

where Vie is used for normalization. Upon completion of 
generate pixel gradient process 603, processing transfers to 
generate texture threshold process 604. 

In generate texture threshold process 604, two different 
processes can be used to generate the threshold that is used 
to determine whether the pixel is an edge pixel. The first 
process is to simply set the threshold to a fixed sufficiently 
large value. The second process utilizes an adaptive edge 
threshold process. In this embodiment, generate texture 
threshold process 604 utilizes an adaptive edge threshold 
process. 

Specifically, the first operation in this embodiment of 
generate texture threshold process 604 is to generate a 
plurality of edge estimators that in this embodiment are pixel 
texture estimators Tk(ij) where k takes on values from one 
to four. A pixel texture estimator is generated for each of the 
four possible edges directions through three pixels-by-three 
pixels window 700, as shown in FIGS. 8A to 8D. Each pixel 
texture estimator is generated by adding absolute values of 
pairwise differences of pixels lying on either side of an edge 
for the four distinct edges. The various pixels in three 
pixels-by- three pixels window 700 are weighted with the 
same weights as in the corresponding operator used to 
generate the spatial gradient, i.e., the weights given in 
expression (3). Weighting based on the Sobel operators has 
been widely tested and reported in the literature as providing 
good performance. This weighting was empirically tested 
and demonstrated to provide good performance. 

For clarity, each of the pixel texture estimators are gen- 
erated as follows. Pixel texture estimator Tl(ij), which 
corresponds to the vertical edge detected by Sobel operator 
HI, is: 

«(/, j) = I Qd- U j - 1) - QV - 1. ; + 1)1 + 

|2G(/,;-i)-2Q(/,y+i)|4.|e(/4-i t y-i)-G{/+i,y+i)| 



Operators Hl(ij) and H2(i j) measure the magnitude of 
horizontal gradient, i.e., a vertical edge 801 (FIG. 8A), and 
the vertical gradient, i.e., a horizontal edge 802 (FIG. 8B), 
respectively whereas operators H3(ij) and H4(i j) measure 65 
the diagonal gradients, i.e., diagonal edges 803 and 804 
illustrated in FIGS. 8C and 8D, respectively. 



If a vertical edge is present through pixel Q(ij), the mag- 
nitude of spatial gradient Gl(i j) is the same as pixel texture 
estimator Tl(i j). 

Pixel texture estimator T2(ij), which corresponds to the 
horizontal edge detected by Sobel operator H2, is: 
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threshold adjustment factor t2 ensures that pixels in textured 
72(( i j) B | Q(i - 1 , j - 1) - Q(i +• l , j - 1) | + areas are classified as textured. 

Alternatively, or additionally generate pixel gradient pro- 
21 Q{i- 1, ;)-(?('■•- 1, »l + 1 Q0- l, ; + U - G(/+ 1, j + Dl cess 603 can be mo dified to mitigate the effects of artificial 

5 gradients along block boundaries. The blocking artifacts 
If a horizontal edge is present through pixel Q(ij), the associated with block boundaries contribute primarily to 
magnitude of spatial gradient G2(ij) is the same as pixel gradients associated with either a horizontal edge or a 
texture estimator T2(ij). vertical edge depending on whether the block boundary is 

Pixel texture estimator T3(i,j), which corresponds to the horizontal or vertical. Thus, in generate pixel gradient pro- 
forty-five degree diagonal edge detected by Sobel operator ]0 ces s 603, the effect of blocking artifacts is mitigated by 
H3, is: ignoring the artificial gradients induced by blocking artifacts 

in generating pixel gradient A(ij) and adaptive texture 

ra(U)= iai./-i)-«/ + i./>| + threshold TEXTHRS(ij). 

For example, spatial gradient Gl(i,j) generated by Sobel 
2\Q(i-\j-l)-Q(i+Uj+l)\ + \Q(i-Lj)-Q(ij + l)\ 35 operator HI to detect a vertical edge is discarded in gen- 
eration of pixel gradient A(ij) along the vertical block 

TjC „ * . . boundaries. Similarly spatial gradient G2(i,i) generated by 

If a forty-five degree diagonal edge is present through pixel Sobel Qr m tQ ^ a ^ m 

QOj), the magnitude of spaUal gradient G3(, d ) is the same generation of pixel gradient A(i,j) along the horizontal block 

as pixel texture estimator T3(i j). Z j - A *i *l ?• * ■ j • - i 

, 4 4 t . 4 , , , • , 20 boundaries. Consequently, the normalization used m pixel 

Pixel textare estimator T4(i,j), which corresponds to the ^ent A(i j) and adaptive texture threshold TEXTHRS(ij) 

b ne sSo erTtor H^is fe ^ M bl °= k COmer$ Where artiQcial h ° riz ° ntal as ™" « 

^ " ' vertical gradients are present, the variable adaptive texture 

threshold, described above, is used. Upon completion of 
T4{i, j) = I Q(i- l, j) - Q(i, j - i) I + 25 generate texture threshold process 604, processing transfers 

2 1 Q (i - 1 , y + 1 ) _ Q (i + 1 , ; _ i ) n- 1 e( /, j + 1) . G (i + 1, ;■> | to determine window intensity process 605. 

In determine window intensity process 605, a background 
luma S(i,j) is estimated using the intensity of the pixels in 
If an one hundred thirty five degree diagonal edge is present three pixels-by-three pixels window 700 surrounding cur- 
through pixel Q(ij), the magnitude of spatial gradient G4(i, 30 rent pixel Q(i j). The background luma is generated because 
j) is the same as pixel texture estimator T4(ij). the perceived contrast of an object by the human eye 

After each of the plurality of pixel texture estimators is depends on the luminance of the background. This charac- 
generated, generate texture threshold process 604 uses the teristic of the human visual systems (HVS) has been docu- 
pixel texture estimators to generate an adaptive texture mented in Weber's law. See, for example, A. K. Jain, 
threshold TEXTHRS(ij). In this embodiment, adaptive tex- 35 Fundamentals of Digital Image Processing, Prentice Hall, 
hire threshold TEXTHRS(ij) is a fraction of the sum of pixel New Jersey, 1989. The difficulty of discerning edges 
texture estimators Tl(i,j), T2(i,j) T3(i,j), and T4(i,j). increases with an increase in the average luminance of the 
Specifically, background. Thus, luminance masking is utilized by edge 

detector 520. 

texthrs{1, j) = {t/ 16) * (77(/, j) + 72(/, j) + 73(/, /) + T4{i, j)) 40 In this embodiment, background luma S(i,j) is generated 

by summing the luminance for each pixel in the 3x3 window 
and dividing by nine. This weighting is used to generate 
where Via is used for normalization, and threshold adjust- background luma S(i j) to provide a measure of the average 
ment factor t is determined as described below. luminance in the 3x3 window. Also, this measure can be 

Transform coded images at low data rates show blocking 45 used directly in Weber's law. 
artifacts. Since block boundaries are low contrast edges, care Upon generation of background luma S(i,j), processing 
is taken to ensure that edge detector 520 does not classify transfers from determine window intensity process 605 to 
blocking artifacts as edges. In one embodiment of edge generate intensity threshold process 606. In generate inten- 
detector 520, variable thresholding is utilized. Specifically, sity threshold process 606, luminance threshold B(ij) is 
a larger adaptive texture threshold TEXTHRS(i j) is used for 50 generated that is proportional to background luma S(i,j). In 
classification of pixels along block boundaries, because the this embodiment, luminance threshold B(i,j) is defined as: 
blocking artifacts result in block boundaries typically having 

higher spatial gradients. Hence, in this embodiment, a first tf(y>max(*min, p*s(ij)) 

threshold adjustment factor tl is used for pixels on transform 

block boundaries, and a second threshold adjustment factor 55 wnere 

t2 is used for all pixels other than those on transform block p«luminance threshold adjustment factor; and 
boundaries. First threshold adjustment factor tl is greater Bmin=minimum luminance threshold, 

than second threshold adjustment factor t2. Minimum luminance threshold Bmin is used to eliminate 

The particular levels selected for first threshold adjust- spurious edges, which are areas where the luminance con- 
ment factor tl and second threshold adjustment factor t2 60 trast is too low for the human eye to perceive an edge. The 
were selected empirically. A representative set of images is level of minimum luminance threshold Bmin is empirically 
processed with various levels of these thresholds. The par- selected to prevent such areas from qualifying as edges, 
ticular levels selected are those which provide performance Luminance threshold adjustment factor p is empirically 
consistent with observations made by human observers. The selected such that pixels which are perceived as edges by the. 
level selected for first threshold adjustment factor tl ensures 65 human eye are appropriately classified. In this embodiment, 
that pixels on block boundaries in smooth areas are not minimum luminance threshold Bmin is taken as sixteen and 
classified as textured whereas the level selected for second luminance threshold adjustment factor p is taken as 0.20. 
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Upon completion of generate intensity threshold process pixels within a three pixels-by-three pixels window sur- 
606, processing transfers to edge identifier process 607. rounding the current pixel. As explained above, a pixel is 
In edge identifier process 607, current pixel Q(i,j) is classified as either from an edge, boundary or "shade" 
identified as an edge pixel if pixel gradient A(i j) exceeds (smooth) area. Pixels from edges are left unchanged. Pixels 
both adaptive texture threshold TEXTHRS(ij) and himi- 5 ^om edge boundaries are directionally filtered with a one- 
nance threshold B(i j) and otherwise as not an edge pixel. dimensional filter, and shade pixels are low pass filtered with 
This completes the operation of edge detector 520 in this a two-dimensional filter. 

embodiment Typically, in prior art post-processing systems, edge 

Each non-replicated pixel in the 18x18 block is processed d ct f ction umts f ol }? w b * ihin ™?> cd * e 

by edge detector 520 as indicated in FIG. 6. However, the io tests ? nd ed S e f . hnku g? * °, btam fi th * ed * e ^^p. These are 

J -n t < 1 c j j\ complex operations. Pixel classification using only the edge 

specific operations within a particular process o edgedetec- a threc pixels . by . three pixels has si |. 

tor 520 can be modified to accommodate factors such as nificaQtl ^ lexil b lmki and nc J y 

tuning constraint, coding data rates, processing power, disjoint cdge ^ mcnis for thc pos t- P rocessing. The results 

texture and significance of artifacts, for example. are similar t0 the results obtained using the more complex 

Specifically, since the complexity of edge detection is an 15 operations 

important criterion in real-time implementations, in one FIG 9 ^ one embodiment of a more detailed process 

embodiment, the process performed by edge detector 520 performed by switched filter unit 330. As explained above, 

was a simplified version of the process given above, that is each pixel in the current block is individually processed in 

referred to herein as a simplified edge detector 520A, switched filter unit 330. 

Specifically, with simplified edge detector 520A, in gen- 20 Initially, edge pixel check 541 determines whether the 

erate spatial gradients process 602, only Sobel operators HI edge flag is set for the current pixel. As explained above, if 

and H2 (See expression (3)) were used to generate horizon- the edge flag is set, the current pixel is from an edge in the 

tal spatial gradient Gl(ij) and vertical spatial gradients image and to maintain the sharpness of the edges, edge 

G2(i j) in place of all four Sobel operators. The use of only pixels are not filtered. Thus, if the edge flag is set for the 

the vertical and horizontal spatial gradients required corre- 25 current pixel, the pixel is classified as an edge pixel, and 

sponding changes in each of the subsequent processes. processing transfers from edge pixel check 541 to copy pixel 

Specifically, in generate pixel gradient process 603, pixel 550 and in turn to last pixel check 547, and otherwise to 

gradient A(i,j) is defined as: classify pixel process 542. 

Classify pixel process 542 analyzes the edge map for a 

^CwX 1 ^) * (1^1(0^1+1^2 30 three pixels-by-three pixels window about the current pixel, 

where (W) is used for normalization. A filter control flag is set equal to the number of pixels in the 

In generate edge threshold process 604, only pixel texture thr< * P^ls-by-three pixels window centered on the current 

estimators Tl(ij) and T2(i,j) are generated as described d ™°t have the ed S e **• ^ this embodiment, 

above. Thus, in this simplified embodiment, adaptive texture „ he filt f ™ ntT ? fla & * m & s fo "° ne t0 Qme / U P on com ^' 

threshold TEXTHRS(ij) is: 35 of cla ^ P 15 f P^cess 542, processing m switched 

niter unit 330 transiers to shade pixel check 543. 
TCXTHRS(vX'/8)*(n(y)+72(w)) Shade pixel check 543 transfers processing to two- 
dimensional filter process 544 if the filter control flag is nine, 
where threshold adjustment factor t is taken as first threshold i.e., if none of the pixels in the three pixels-by-three pixels 
adjustment factor tl and second threshold adjustment factor 40 window have the edge flag set so that, the current pixel is a 
tZ First threshold adjustment factor tl is used for pixels on shade pixel. Ensuring that all pixels in the three pixels-by- 
block transform boundaries and is taken as 0,90. Second three pixels window are not edge pixels implies that all 
threshold adjustment factor t2 is used for all pixels other pixels processed by twoKlimensional filter process 544 are 
than those on block transform boundaries and is taken as on the same side of an edge and do not either span or include 
0.80. The remaining processes in edge detector 520A are the 45 an edge. Therefore, edges are not smeared by two- 
same as those described above. dimensional filter process 544. If the filter control flag is less 

When tested on sequences, this simplified edge detector than nine, shade pixel check 543 transfers processing to 
generated accurate edge maps identifying areas which would continuous edge check 901, that is described more com- 
be classified as edges by the human eye. Nevertheless, the pletely below. 

above embodiment of simplified edge detector 520A is only 50 In this embodiment, two-dimensional filter process 544 
one of many embodiments available for edge detection unit utilizes a two-dimensional low pass filter. The low pass filter 
320. Simplifications and modifications to edge detection unit is a separable three -tap low pass filter (LPF) in each direc- 
320 can be made for specific system requirements. Such tion with a mask of the form: 
simplifications could include replacement of adaptive tex- 
ture threshold TEXTHRS(ij) by a fixed threshold and 55 
elimination of luminance masking through eliminating the 
test with luminance threshold B(i,j). 

When a block has been processed by edge detection unit 
320, an edge map is stored for the current block and the one 

pixel boundary around the current block in edge-block 60 
memory 430. The edge map, as described above, is a 
plurality of edge flags where for each pixel, the edge flag is 

set if the pixel is on an edge and is cleared if the pixel is not Two-dimensional filter process 544 reduces the mosquito 

on an edge. In this embodiment, switched filter unit 330 uses noise and the blocking artifacts in image areas at a distance 

the edge map without further processing. 65 greater than or equal to one pixel horizontally and vertically 

Specifically, switched filter unit 330 determines the fil- from edge pixels. The filtered output pixel from the two- 

tering applied to each pixel by examining the edge map for dimensional filter is written to the appropriate location in 
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sixteen pixels-by-sixteen pixels filter block memory 425, as 
explained above, and processing transfers to last pixel check 
547. 

Recall that if the filter control flag is less than nine, shade 
pixel check 543 transfers processing to continuous non-edge 5 
check 901. Continuous non-edge check 901 determines 
whether enough non-edge pixels are present in the three 
pixels-by-three pixels window including the current pixel to 
perform further directional -filtering without filtering an edge 
pixel. Specifically, if at least three of the pixels are non-edge 10 
pixels, a continuous line of non-edge pixels may exist 
through the window. However, if less than three of the pixels 
in the current window are not-edge pixels, any one- 
dimensional filter along any possible axis through the cur- 
rent window would include an edge pixel. Consequently, if 15 
the filter control flag is less than three, continuous non-edge 
check 901 transfers processing to copy pixel process 950 
and otherwise to vertical line of non-edge pixels check 902. 
Copy pixel process 950 copies the pixel into the proper 
location in filter block memory 425 and transfers processing 20 
to last pixel check 547. 

Vertical line of non-edge pixels check 902 determines 
whether the current window includes a vertical line of 
non-edge pixels through the current pixel. Specifically, FIG. 
10A illustrates the current window configuration in the edge 25 
pixel map that is detected by vertical line of non-edge pixels 
check 902. If the pixel immediately above current pixel 
1001, the pixel immediately below current pixel 1001 are 
non-edge pixels, vertical line of non-edge pixels check 902 
is true and otherwise false. 30 

In FIGS. 10A to 10D, a non-edge pixel is represented by 
a zero to indicate that the edge flag is cleared for that pixel. 
Pixels that are not on the vertical axis through current pixel 
are represented by an "x", because the state of the edge flag 
for these pixels is a don't care state. If vertical line of 35 
non-edge pixels check 902 is true, processing transfers to 
one-dimensional vertical filter process 903 and otherwise to 
horizontal line of non-edge pixels check 904. 

In one-dimensional vertical filter process 903, the vertical 
line of non-edge pixels in the current window is filtered by 40 
a three-tap filter. In this embodiment, the three pixels are 
processed by a filter of the form [V*, Yz y Ya] and the filtered 
output pixel from the one-dimensional filter is written to the 
appropriate location in sixteen pixels-by-sixteen pixels filter 
block memory 425, as explained above. Upon completion of 45 
the filtering process, one-dimensional vertical filter process 
transfers 903 to last pixel check 547. 

Horizontal line of non-edge pixels check 904 determines 
whether the current window includes a horizontal line of 
non-edge pixels through the current pixel. Specifically, FIG. 50 
10B illustrates the current window configuration in the edge 
pixel map that is detected by horizontal line of non-edge 
pixels check 904. If the pixel immediately to the left of 
current pixel 1001, the pixel immediately to the right of 
current pixel 1001 are non-edge pixels, horizontal line of 55 
non-edge pixels check is true and otherwise false. If hori- 
zontal line of non-edge pixels check 904 is true, processing 
transfers to one-dimensional horizontal filter process 905 
and otherwise to forty-five degree line of non-edge pixels 
check 906. 60 

In one-dimensional horizontal filter process 905, the 
horizontal line of non-edge pixels in the current window are 
filtered by a three-tap filter. In this embodiment, the three 
pixels are processed by a filter of the form [V4, l A t V4] and the 
filtered output pixel from the one-dimensional filler is writ- 65 
ten to the appropriate location in sixteen pixels-by-sixteen 
pixels filter block memory 425, as explained above. Upon 



completion of the filtering process, one-dimensional hori- 
zontal filter process transfers to last pixel check 547. 

Forty-five degree line of n on -edge pixels check 906 
determines whether the current window includes a forty-five 
degree line of non-edge pixels through current pixel 1001. 
Specifically, FIG. 10C illustrates the current window con- 
figuration in the edge pixel map that is detected by forty-five 
degree line of non-edge pixels check 906. If the pixel 
diagonally above and to the left of current pixel 1001 and 
pixel diagonally below and to the right of current pixel 1001 
are non-edge pixels, forty-five degree line of non-edge 
pixels check 906 is true and otherwise false. If forty-five 
degree line of non-edge pixels check 906 is true, processing 
transfers to one-dimensional forty-five degree filter process 

907 and otherwise to one hundred thirty- five degree line of 
non-edge pixels check 908. 

In one -dimensional forty-five degree filter process 907, 
the forty-five degree line of non-edge pixels in the current 
window are filtered by a three-tap filter. In this embodiment, 
the three pixels are processed by a filter of the form [V4, Vi, 
l A] and the filtered output pixel from the one -dimensional 
filter is written to the appropriate location in sixteen pixels- 
by-sixteen pixels filter block memory 425, as explained 
above. Upon completion of the filtering process, one- 
dimensional forty-five degree filter process 907 transfers to 
last pixel check 547. 

One hundred thirty-five degree line of non-edge pixels 
check 908 determines whether the current window includes 
a one hundred thirty-five degree line of non-edge pixels 
through current pixel 1001. Specifically, FIG. 10D illustrates 
the current window configuration in the edge pixel map that 
is detected by one hundred thirty-five degree line of non- 
edge pixels check 908. If the pixel diagonally below and to 
the left of current pixel 1001 and the pixel diagonally above 
and to the right of current pixel 1001 are non-edge pixels, 
one hundred thirty-five degree line of non-edge pixels check 

908 is true and otherwise false. If one hundred thirty-five 
degree line of non-edge pixels check 908 is true, processing 
transfers to one -dimensional one hundred thirty-five degree 
filter process 909 and otherwise to copy pixel process 551 
that writes the current pixel to filter block memory 425 and 
then transfers to last pixel check 547. 

In one-dimensional one hundred thirty-five degree filter 
process 909, the one hundred thirty-five degree line of 
non-edge pixels in the current window are filtered by a 
three -tap filter. In this embodiment, the three pixels are 
processed by a filter of the form [Vfc, Vi] and the filtered 
output pixel from the one-dimensional filter is written to the 
appropriate location in sixteen pixels-by-sixteen pixels filter 
block memory 425, as explained above. Upon completion of 
the filtering process, one-dimensional one hundred thirty- 
five degree filter process 904 transfers to last pixel check 
547. 

Last pixel check 547 determines whether all the pixels in 
the current block have been processed by switched filter unit 
330. If one or more pixels remain for processing, last pixel 
check 547 transfers to edge pixel check 541 and otherwise 
to copy filtered block process 548 that was described above. 

Thus, in this embodiment of switched filter unit 330 pixels 
directly adjacent to edges are not processed with the two- 
dimensional filter because such a filter would include pixels 
from either sides of an edge. However, it is desirable to clean 
up the area next to edges (edge border areas) to the maxi- 
mum extent possible without smearing the edges since this 
results in clean sharp edges which are critical to the per- 
ceived quality of an image. Thus, as explained above if at 
least three pixels in the window including the current pixel 
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are not edge pixels, the window is examined to see if all the by only processing the luminance component. This choice is 

pixels lying along one of the four possible axes through the reasonable since at low to moderate data rates, luma varia- 

window are not edge pixels. If an axis is made-up of tions mask the chroma artifacts. 

non-edge pixels, the pixels on that axis are processed with . Artifact reduction unit 350 was used by itself on a variety 

a one-dimensional filter. 5 0 f test sequences that included sequences at QCIF resolution 

Notice that the four possible axes are checked sequen- and sequences generated at QCTX resolution. The QCIF 

tiaily and the directional filtering is performed along the first format is a standard format employed in international stan- 

axis along which all the pixels are not-edge pixels. Axis dards such as jj 261 and H.234. The QCTX format is a 

examination is stopped after the first axis along which proprietary format employed in the AT&T 2500 videophone, 

filtering is allowed ls found. Although adjacent pixels are not 1Q ^ QCJ ^ sequences Vre coded using a PV2 video coder, 

examined for continuity of direction, axis examination . . ^ . . A ^ 0 %«™. . 

always proceeds in the order shown above. This ensures that * al * the » me Vlde0 c ^ er ** *f 2500 videophone 

adjacent pixels are classified similarly if ambiguity in clas- at data rates ran S m 8 &om 10 „ t ? -c 20 k !* s - ™ e QCIF 

sification exists. sequences were coded using a H.261 coder at data rates 

This process of axis selection and resultant one- ranging from 48 to 112 kbps. Maximum frame rate was set 

dimensional directional filtering is equivalent to finding 15 to 10 frames per second. All of these sequences are suitable 

pixels adjacent to the edges, i.e., finding edge border areas, for personal video communications. In order to ensure that 

and filtering pixels in the edge border areas along a direction artifact reduction unit 350 does not introduce any degrada- 

parallel to the edges. This technique also provides edge tions in more complex sequences, artifact reduction unit 350 

enhancement. was applied to a widely used test sequence named Mobile at 

Edge pixels and pixels directly adjacent to the edge pixels 20 CIF resolution coded open loop with a fixed quantizer scale 

that are not selected for one-dimensional directional filtering of 32 at 10 frames per second with a H.261 coder, 

are not post-processed. Leaving these pixels unchanged Artifact reduction unit 350 was successful in eliminating 

ensures that sharpness of edges in the decoded image is not most of the coding artifacts while preserving edges. The 

degraded by post-processing. This implies that the pixels visual improvement in the quality of low to medium bit rate 

adjacent to edges which da not qualify for one -dimensional 25 coded images was striking. 

directional filtering are also treated like edges. This is These extensive simulations show that the artifact reduc- 

equivalent to coalescing edge segments with a small dis- tion unit 350 substantially and dramatically improves the 

continuity (1 to 2 pixels) into continuous contours. performance of low to medium bit rate video codecs by 

Therefore, the pixel classification process in switched filter cleaning coding artifacts while preserving edge sharpness, 

unit 330 compensates for the lack of edge linking and tracing 30 The post-processor 300 presented is independent of the 

and allows post -processor 300 to effectively use an edge encoder. Therefore post-processor 300 can be added to any 

map equivalent to those generated by more complex edge video receiver between decoder and display modules to 

detectors. enhance the displayed image. 

As is known to those skilled in the art, color video In one embodiment of this invention, the various pro- 
sequences typically consist of three color components, for 35 cesses described herein are written in the C++ computer 
example, red, green and blue components. Current coding language and are compiled and linked using the CenterLine 
standards, such as H.261 and H.262, use different color CC compiler to obtain binary code that can be stored in a 
components. These standards use one luminance component non-volatile memory of a decoder. In one embodiment, the 
which carries the gray-scale, or average image intensity binary code was used with a simulated decoder and the 
corresponding to a black and white video sequence and two 40 output images from post-processor 300 were stored and 
chroma components which contain the color information. In subsequently displayed using video players, 
this embodiment, color video sequences are post-processed While post-processor 300 provides a significant enhance - 
by applying the post-processor to the luminance component. ment in image quality, further enhancements in both picture 
At low data rates, chroma components show severe block- quality and performance are obtained by taking advantage of 
ing. When edge-detector 520 is applied to the chroma 45 knowledge of one or more coding parameters from the 
components, no edges are detected since the coded chroma decoder, such as the quantizer scale, for pixel filtering. As 
has no distinct edges. In the absence of edges, all pixels explained more completely below, utilizing information 
appear as shade pixels and so post-processor 300 reduces to about a coding parameter, such as the quantizer step size in 
a two-dimensional low pass filter. the post-processor of this invention introduces coupling 

Some improvement in chroma noise is observed by low 50 between the post-processor and the coder, 
pass filtering the chroma with the two-dimensional low pass This embodiment of a coding parameter adaptive post- 
filter. Specifically, the filter mask is: processor adapts to the coding parameters available at the 

decoder, e.g., a quantizer scale is used. Since the quantizer 
scale is indicative of the degree of quantization, the quan- 
55 tizer scale also is indicative of the amount of post -processing 
that is needed. In addition, to using the quantizer scale in the 
post-processing, the components in this embodiment of the 
quantizer scale adaptive post-processor have been simplified 
to assist in real-time implementation. 
60 As is known to those skilled in the art, current coding 
standards such as H.261 and MPEG first apply a DCT to 
Both chroma components are low pass filtered with this non -overlapping square blocks in the current frame to trans- 
filter. The improvement in chroma noise is more pronounced form the spatial domain into a transform domain. The 
for a coded video rate of 96 kbps as opposed to 56 or 4S resulting transform coefficients for each block are quantized 
kbps. 65 using a matrix W and quantizer scale q. The quantized value 
Since the improvement is minimal for the lower video Q(C(ij)) of transform coefficient C(ij) in the ith row and jth 
rates, the complexity of post-processor 300 can be reduced column of the block is: 
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Since the horizontal and vertical spatial gradient factors 

C[i, j) are stored, the memory requirements of post-processor 1100 

Q(C(i. ;)) - ^ ^ x ^ are j ncrease( j relative to those of post-processor 300. 

However, the greater memory requirements are offset by the 
5 enhanced speed of post-processor 1100 relative to post- 
Typically, quantizer step size q is defined as: processor 300. 

q-comfQS After the horizontal and vertical spatial gradient factors 

are generated by edge pre-processor 1121, the factors are 

wnere used by edge detection unit 1122 to generate pixel-by-pixel 

QS=quantizer scale; and 30 a horizontal spatial gradient and a vertical spatial gradient. 

const-2. The horizontal spatial gradient and vertical spatial gradient 

Therefore, the quantizer scale is a direct measure of the for a pixel are combined to form a pixel gradient. The pixel 

coarseness of quantization. gradient is compared with a single adaptive edge threshold 

As demonstrated above, post -processing reduces coding to determine whether the pixel is an edge pixel. If the pixel 

artifacts. If quantization errors are minimal, coding artifacts 15 is an edge pixel, an edge flag for the pixel is set in an edge 

are also minimal, and therefore the post-processing should map stored in edge-strip memory 1107. Consequently, a 

be minimal. However if quantization errors are large, the pixel -by-pixel edge map is generated and stored in edge 

coding artifacts are severe and the sequence should be memory 1107 by edge detection unit 1120. In this 

heavily post-processed. This embodiment of quantizer scale embodiment, the use of a single adaptive edge threshold in 

adaptive post -processor, as described more completely 20 the edge detection process further simplifies edge detection 

below, utilizes digital noise reduction, edge detection, unit 1122 by replacing the two distinct thresholds used in 

switched filtering and a quantization adjustment. edge detection unit 320 with a single threshold. 

FIG. 11 is a block diagram of one embodiment of quan- Switched filter unit 1131 in filter processor 1130 again 

tizer scale adaptive post-processor 1100. Post -processor classifies pixels as one of an edge pixel, an edge boundary 

1100 utilizes a simplified edge detection unit 1122 and 25 pixel, and a shade pixel using the edge map for a three 

switched filter unit 1130 compared to the embodiments pixel -by -three pixel window about the current pixel. As 

described above to assist in real-time implementation. described above, edge pixels are not further processed. Edge 

Initially, post-processor 1100 updates a quantizer scale boundary pixels are filtered with an appropriate one dimen- 
map for a frame in input frame memory 1102 from the sional filter, and shade pixels are filtered with a two- 
decoder. In this embodiment, the quantizer scale map is 30 dimensional low pass filter. Any non-edge pixels that are not 
transferred a frame at a time. Alternatively, a portion of the classified as either a shade or edge-boundary pixel are not 
frame quantizer scale map could be transferred on a mac- processed by switched filter unit 1131, This process is also 
roblock by macroblock basis. The quantizer scale map, that simplified by utilizing the stored horizontal and vertical 
is stored in a quantizer scale memory 1108 of post-processor spatial gradient factors, as described more completely 
1100, has one entry, i.e., a quantizer scale, for each mac- 35 below. 

roblock in the frame. The quantizer scale for a macroblock After a pixel is processed by switched filter unit 1131, a 
in a new frame is updated only if that macroblock is not quantizer scale adaptation unit 1132 utilizes a quantizer 
replenished. Here, replenished means that the macroblock in scale adaptive filter to adjust the post-processed output pixel 
the current frame is obtained by copying the decoded from switched filter unit 1131 for the quantizer scale, 
macroblock at the same spatial location from the immedi- 40 Specifically, the extent of post-processing is adjusted in 
ately preceding decoded frame. Thus, if the macroblock is quantizer scale adaptation unit 1132 by mixing the post- 
replenished, the quantizer scale of the corresponding mac- processor output pixel from quantizer scale adaptation unit 
roblock in the previous frame is used. 1132 with the digitally noise reduced output pixel from the 
As explained more completely below, the quantizer scale decoder. As explained more completely below, quantizer 
is used in a quantizer scale adaptive filter. Thus, specification 45 scale adaptation unit 1132 does not use a quantizer scale 
of the quantizer scale stored in the quantizer scale map for directly, but rather uses a weighting factor that is propor- 
a particular macroblock determines the performance of the tional to quantizer scale QS for the macroblock containing 
filter for that macroblock. Thus, in this embodiment, char- the current pixel. The filtered output pixel from filter pro- 
acteristics of a macroblock are used to adapt the quantizer cessor 1130 is the weighted sum of the digitally noise 
scale adaptive filter to that macroblock. In view of this 50 reduced pixel and the post-processed pixel, 
disclosure, other characteristics of the macroblock could be Post-processor 300 had a mechanism for identifying sta- 
used to adapt a coding parameter adaptive filter for a tionary textured blocks which were replenished. These 
particular macroblock. blocks were not post-processed in order to preserve sharp- 
Digital noise reduction unit 1110 is similar to digital noise ness. Thus, post-processor 300 did not need any parameters 
reduction unit 310 except the replenishment flag is not 55 from the decoder. In the current implementation, the quan- 
needed. In this embodiment, artifact reduction unit 1150 tizer scale adaptation, i.e., a coding parameter adaptation, 
includes an edge processor 1120 and a filter processor 1130. preserves sharpness in well coded areas by minimizing the 
Edge processor 1120 includes an edge detection unit 1122, extent of post-processing. Stationary areas in particular are 
and an edge pre-processor 1121, Edge pre-processor 1121 likely to be replenished. If the stationary areas.are quantized 
initially generates weighted three pixel horizontal and ver- 60 finely in a previous frame, the replenished block is close to 
tical spatial gradient factors around a pixel and stores the the original and therefore should not be post -processed, 
horizontal and vertical spatial gradient factors for subse- Quantizer scale adaptive post-processor 1100 preserves the 
quent use in horizontal sum memory 1106 and vertical sum sharpness for replenished macroblocks if the macroblocks 
memory 1105, respectively. As explained more completely were quantized using a small step-size in previous frames by 
below, the horizontal and vertical spatial gradient factors are 65 using the quantizer scale from the previous frames for the 
used repeatedly in sub-processes within edge detection unit replenished macroblocks. This eliminates the need for iden- 
1122 and switched filter unit 1131. tification and separate processing of stationary textured 
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blocks as in post-processor 300. Therefore, post-processor present value in the map is replaced with the quantizer scale 

1100 preserves sharpness by quantizer scale adaptation for the current macroblock. Update quantizer scale process 

instead of by textured stationary block identification which 1203 also transfers processing to update block 1204. 

is eliminated. Thus, when processing transfers to update block 1204, the 

Quantizer scale adaptive post-processor 1100 relies on the 5 quantizer scale at location (i,j) in quantizer scale map 

decoder to provide a measure of the quality of encoding QSMap_n(ij) is changed only if the macroblock is not a 

(quantizer scale) to post-processor 1100, Thus, the decoder replenished macroblock. Update block process changes indi- 

and the post-processor are coupled to some extent. However, ces (ij) to point to the next block in the frame and transfers 

for present codecs this data is readily available, and so this processing to frame complete check 1205. 

is not a significant issue. no Frame complete check 1205 determines whether all the 

Quantizer scales are transmitted from the encoder over the macroblocks in the frame have been processed. If all the 

communication channel to the decoder and are therefore macroblocks have been processed, frame complete check 

available in the receiver. In most video-conferencing codecs, 1205 transfers processing to load memory process 1206 and 

the quantizer scale is fixed for a macroblock that typically otherwise to replenished check 1202. Thus, each quantizer 

consists of sixteen pixels by sixteen pixels. Typically, 15 scale in quantizer scale map QS Map__n(ij) is updated or left 

replenished macroblocks and not-replenished macroblocks unchanged when processing reaches load memory process 

are easily distinguished. A replenished macroblock has all 1206. 

quantized coefficients equal to zero. This is possible since In load memory process 1206, quantizer scale map 
video codecs employ temporal prediction and code a pre- QSMap_n(ij) is transferred from the decoder to post- 
diction error rather than the original pixel value. If the 20 processor U00. The particular type of transfer and the 
current macroblock was represented in the previous frame timing of the transfer are determined by the interface 
with sufficient accuracy and/or the quantizer scale is high between the decoder and post-processor 1100. The impor- 
enough, the prediction error yields all zeroes when quan- tant aspect is that prior to post-processor 1100 starting to 
tized. For such blocks the quality of coding is the same as process a frame, the quantizer scales for the frame are 
in the previous frame. Therefore, the quantizer scale for the 25 available in post-processor 1100. 

previous frame is an accurate measure of the quality of Upon transfer of quantizer scale map QSMap_n(ij) to 

coding. post-processor 1100, update temporal index process incre- 

To utilize quantizer scale adaptive post-processor 1100 ments temporal index n and transfers processing to sequence 

with a particular decoder, the decoder is modified to perform complete check 1208. If temporal index n is less than 

process 1200 (FIG. 12.). The decoder records quantizer 30 maximum frame number N, check 1208 transfers to copy 

scales for each frame in a two dimensional array QSMap_ process 1210 and otherwise to done 1209, 

n(ij) in a memory of the decoder, where n denotes the In copy process 1210, quantizer scale map QSMap__n-l 

temporal index of the input frame in input frame memory (ij) is copied to quantizer scale map QSMap__n(i j). This 

1102 and (ij) are the row and column indices of the current process may not be necessary. If quantizer scale map 

macroblock within the input frame. For a frame having 35 QSMap_n-l(ij) is always completely transferred to post- 

NROWS pixel rows and NCOLS pixel columns, and a processor 1100 before the earliest possible time that the 

macroblock size of sixteen pixels-by-sixteen pixels: decoder can start to update the map for the next frame, only 

a single quantizer scale map must be stored at any given 

(0 J 0)S(vKJVROWS/l6 > AfCOl^/i6). otherwise, at least two quantizer scale maps are 

where (x,y)<(a,b) implies that (x<a) and (y<b). Also, tem- 40 ™^ Cd " mem0ry l ° preVeDt ** P 0881 ™ 1 ^ <* «V 
poral index n is defined as: overwriting. ' 

Copy process 1210 transfers to reset indices process 1211. 
o^n<N Reset indices process 1211 resets indices (ij) to (0,0) and 

transfers processing to replenished check 1202 to start the 
where N is the number of frames in the video sequence. 45 processing for the next frame in the sequence. 

Specifically, within process 1200 in initialization process Process 1200 is illustrative only of one embodiment of a 
1201, each location in two dimensional quantizer scale map method for providing quantizer scales to post-processor 
QSMap_n(ij) for temporal index n equal to zero is set to the 1100 and is not intended to limit the invention to the 
lowest possible value, e.g., the minimum value of the particular embodiment described. The important aspect is 
quantizer scale. For a H.261 codec, each location in two 50 that when a frame is available for use in input frame memory 
dimensional array QSMap_0(ij) is set to one and then 1102 (FIG. U) of post-processor 1100, quantizer scales for 
indices i and j are set to zero. Processing transfers from each block in the frame are stored in quantizer scale memory 
initialization process 1201 to replenishment check 1202. 1108. 

In check 1202, process 1200 determines whether the In the embodiment, an input frame is denoted by Fn where 
current macroblock, i.e., the macroblock in the frame having 55 n is the temporal index. As explained above, temporal index 
indices (ij) is replenished. Although this can be done by n can take on any value in the range: 
examining the pixel-to-pixel changes between adjacent Q£ 
frames, in most standard codecs, whether a macroblock is 

replenished can be determined by examining the transmitted Spatial coordinates (i,j) for pixels within frame Fn are in the 
bit-stream. In this embodiment, process 1200 determines 60 range: 

whether a macroblock is replenished by examining the (o.o^ft^owsjvcoLS) 
macroblock layer in the transmitted bit-stream. If the mac- 
roblock is replenished, processing transfers to update block where 

process 1204 and conversely to update quantizer scale NROWS«number of rows in frame Fn; and 
process 1203. 65 NCOLS«number of columns in frame Fn. 

In update quantizer scale process 1203, the quantizer scale Thus, the input to post-processor 1100 is a sequence of N 
in quantizer scale map QSMap_n(i j) is updated, i.e., the frames of size NROWSxNCOLS. As explained more com- 
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pletely below, post -processor 1100 generates a sequence of frame is not processed by digital noise reduction unit 1110 

N post-processed frames of size NROWSxNCOLS. and artifact reduction unit 1150. 

Post-processor 1100 includes a reference frame memory After the zeroth frame is processed in initialize memory 

1101, a current frame memory 1103, an in-strip memory process 1401, post-processor U00 transfers to update tem- 

1104, a vertical sum memory 1105, a horizontal sum 5 P°ral index process 1402. Update temporal index process 

memory 1106, an edge-strip memory 1107, a quantizer scale 1402 changes the value of temporal index n to indicate the 

memory 1108, an output-strip memory 1109 and an output ncxt frame that is processed by post-processor 1100 and then 

frame memory 1140. While these memories are shown in transfers processing to sequence complete check 1403. 

FIG. 11 as separate distinct memories, those skilled in the art In sequence complete check 1403 post-processor 1100 

will appreciate that the memories can be diiferent regions in 10 com P ares < e f m P oral m , dex ? Wlth number of frames N in the 

one or more memories as well as separate memories. "3^^ n * f 15 greate f r K than or ^ t0 

¥ . . . uni j * number of frames N, all the frames in the sequence have 

id uus emDoaimem, current memory m» ana reierence ^ ^.^^ and 

so processing transfers to done 

frame memory 1101 are two pixels wider than the input ^ 1404 which fa tum ^ rforms ^ tions neces . 

frame along all borders. As explained more completely sary t0 terminate 0 p 6ra tions of post-processor 1100 for the 

below, wider memories are needed since post-processing for 15 video sequence. Conversely, if temporal index n is less than 

each pixel requires edge information for pixels in a three number of frames N, all the frames in the sequence have not 

pixel-by-three pixel window surrounding the current pixel. been post-processed and so processing transfers to frame 

Therefore, current frame memory 1103 and reference frame copy process 1405 

memory 1101 are four pixels larger in each direction than In frame copy process 1405, the data in input frame 
input frame memory 1102. 20 memory 1102 is copied into the center of current frame 
FIG. 13 is an illustration of the size of current frame memory U03, i.e., so that the frame of data is surrounded by 
memory 1103 and reference frame memory 1101. Each a two pixel border. In this embodiment, the quantizer scale 
memory has PCOLS columns and PROWS rows. When the map is updated during decoding and a frame is post- 
size of the frame is NROWS by NCOLS, PCOLS and processed only after the frame has been fully decoded. Since 
PROWS are defined as: 25 decoding is completed before post-processing is initiated, 

there is no need to check that the quantizer scale map is 

FROWS=tfROWS+4; and „ , f A • tU - u 4 u n . , 4 , , 

updated in this embodiment. However, in general, the post- 

pcols=jvcols+4. processor should ensure that the quantizer scale map is 

updated. Upon completion of the copy, processing transfer 

As shown in FIG, 13, the input frame is copied to the center 30 to initialize strip index process 1406. 

of the applicable frame memory in post-processor 1100. Initialize strip index process 1406 sets the value of an 

As explained more completely below, the data in current mitial row strip pointer STRIP_START to a predetermined 

frame memory 1103 are processed a strip at a time. In this vanje; e g > ^0 ^ predetermined value is selected to that 

embodiment, the data in current frame memory 1103 are initial row strip pointer STRIP_START identifies the first 

divided into overlapping horizontal strips with an overlap of 35 ro W m t he frame that is processed by post-processor 1100. 

two pixels at the top as well as the bottom of the strip. u pon initialization of initial row strip pointer STRIP_ 

As illustrated in FIG. 13, the width of a strip is SROWS START, process 1406 transfers processing to frame corn- 
rows and the length of a strip is SCOLS columns. SROWS p i c t e check 1407. 

rows is greater than four rows and less than or equal to the In complete check 1407, post-processor 1100 corn- 
total number of rows in the frame NROWS, i.e., 40 pares ioiti&L row strip pointer STRIP_START with number 
4<SROWS§A r ROWS. °^ rows NROWS in the current frame. If initial row strip 

pointer STRIP__START is greater than or equal to number of 

In the embodiment described more completely below, the rows NROWS, the post-processing of the frame is complete 

value of SROWS was twenty. Also, it was assumed that the and so processing transfers to copy frame process 1473. 

total number of rows in the frame was divisible by 45 Copy frame process 1473 copies the frame of post-processed 

(SROWS-4). This assumption ensures that the frame can be pixel data from output frame memory 1140 to reference 

divided into equal sized strips. If this assumption is not true, frame memory 1101 and transfers processing to update 

slight modifications are needed to post -process the residual temporal index process 1402, that was described more 

lines at the end of the frame. completely above. Conversely, if initial row strip pointer 

SCOLS, in this embodiment, is taken equal to PCOLS. 50 STRIP_START is less than number of rows NROWS, all 

Since only a strip of the current frame is processed at a time, the rows in the frame have not been post-processed and so 

in-strip memory 1104, vertical sum memory 1105, horizon- processing transfers to strip copy process 1408 

tal sum memory 1106, output -strip memory 1109 and edge- In strip copy process 1408, the data in each row starting 

strip memory 1107 are SROWS rows wide and SCOLS with row STRIP_START and ending with row (STRIP_ 

columns long. 55 START+SROWS) are copied from current frame memory 

FIGS. 14A to 14G are a process flow diagram of one 1103 to in-strip memory 1104. 

embodiment of the operations performed by the various Upon completion of strip copy process 1408, all the 

units of post-processor 1100 (FIG. 11). When post-processor necessary initialization for processing of a strip is complete, 

1100 starts to process a sequence of N frames, the zeroth and so digital noise reduction unit 1110 initiates processing 

frame is written to input frame memory 1102 by the decoder. 60 of the information in in-strip memory 1104, in this embodi- 

Initialize memory process 1401 detects that a new sequence ment. Of course, if digital noise reduction is either unnec- 

is starting and initializes temporal index n to zero. Process essary or unwanted, digital noise reduction unit 1110 could 

1401 also sets the four extra rows and four extra columns of be eliminated. In this case, artifact reduction unit 1150 

memories 1103 and 1101 to a predetermined state, e.g., zero. would initiate processing of the information in in-strip 

Finally, process 1401 copies the zeroth frame from input 65 memory 1104, as described more completely below, 

frame memory 1102 to output frame memory 1140 and to the One embodiment of the process performed by digital 

center of reference frame memory 1101. Thus, the zeroth noise reduction unit 1U0 is illustrated in FIG. 14B. Initialize 
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pixel pointer process 1410 sets a pointer to a pixel in in-strip first pixel processed by edge pre-processor 1121. In this 

memory 1104 to the first pixel processed by digital noise embodiment, initialize pixel pointer process sets indices (i,j) 

reduction unit 1110. In this embodiment, initialize pixel of the pixel pointer to location (1,0) in in-strip memory 1104, 

pointer process 1410 sets indices (i j) of the pixel pointer to since the second row is the first row having a row of pixels 

location (2,2) in in-strip memory 1104, since the first two 5 on each side. Upon completion of initialize pixel pointer 

rows are repeated and the first two columns are not actual process 1420, processing transfers to strip complete check 

pixel data. Upon completion of initialize pixel pointer pro- 1421. 

cess 1410, processing transfers to strip complete check 1411 . Strip complete check 1421 determines whether any pixels 

Strip complete check 1411 determines whether any pixels remain in in-strip memory 1104 that require processing by 
remain in in-strip memory 1104 that require processing by no edge pre-processor 1120 to generate a vertical spatial gra- 
digital noise reduction unit 1110. In this embodiment, the dient factor. In this embodiment, the last pixel processed is 
last pixel processed is at location (SROWS-2, SCOLS-2) at location (SROWS-1, SCOLS) because row SROWS-1 is 
because the last two rows are either repeated in the next the last row that has a row of pixels on each side. Thus, in 
strip, or are not actual pixel data and the last two columns this embodiment, edge pre-processor 1121 initially pro- 
are not actual pixel data. Thus, in this embodiment, digital 15 cesses a strip that is (SOWS -2) rows (FIG. 13) wide and 
noise reduction unit U10 processes a strip that is RROWS SCOLS columns long. If indices (i j) of the pixel pointer are 
rows (FIG. 13) wide and NCOLS columns long. If indices less than (SROWS-1, SCOLS) processing transfers to gen- 
(i j) of the pixel pointer are less than (SROWS-2, SCOLS-2), erate vertical spatial gradient factor process 1422 and oth- 
processing transfers to generate pixel difference process erwise to initialize pixel pointer process 1425. 
1412 and otherwise to initialize pixel pointer process 1420 20 Generate vertical spatial gradient factor process 1422 
(FIG. 14C) in edge pre-processor 1121. generates a three pixel weighted vertical spatial gradient 

Generate pixel difference process 1412 generates digital factor for current pixel Qn(i,j). Specifically, in this 

pixel difference dn(ij), as defined above, using current pixel embodiment, generate vertical spatial gradient factor pro- 

Qn(ij) from in-strip memory 1104 and corresponding pixel cess 1422 generates a weighted three pixel vertical spatial 

Qn_l(STRIP_START+i, j) from reference frame memory 25 gradient factor VERSUM (ij) that is defined as: 

1101. Thus, the two pixels are retrieved from the respective , „ , 

memories and digital pixel difference dn(ij) is formed. w/ * v */ w/ * \ »j 

Upon generation of digital pixel difference dn(i,j) processing where Qn(i,j) is a pixel at location (ij) in in-strip memory 

transfers to access look-up table process 1413. 1104. The relationship between the three pixels used to 

In this embodiment, Table 1, as defined above, is stored in 30 generate weighted vertical spatial gradient factor VERSUM 

a look-up table memory of post-processor 1100. Thus, in (ij) is illustrated in FIG. 15. Specifically, pixel Qn(i-1 j) is 

process 1413, digital pixel difference dn(i,j) is used as an immediately above current pixel Qn(ij) and pixel Qn(i-lj) 

index to access the appropriate value of digital noise reduc- is immediately below the current pixel. In FIG. 15, some 

tion function f(dn) in the look-up table memory. Upon pixels are indicated by an X to show that the pixel processed 

completion of access look-up table process 1413, processing 35 are located within in-strip memory 1104. 

transfers to output DNR pixel process 1414. Upon completion of generate vertical spatial gradient 

In output DNR pixel process 1414, output pixel Qn(i,j) is factor process 1422, processing transfers to store vertical 

generated according to expression (1) above and loaded in spatial gradient factor 1423. Store vertical spatial gradient 

the appropriate location of in-strip memory 1104. Thus, the factor process 1423 stores weighted vertical spatial gradient 

pixel in in-strip memory 1104 is replaced by a noise reduced 40 factor VERSUM(i j) in vertical sum memory 1105 at loca- 

pixel and processing transfers to update pixel pointer pro- tion (ij). Notice that only SROWS-2 by SCOLS are used in 

cess 1415. vertical sum memory 1105, which, as described above, has 

Update pixel process 1415 changes indices (ij) of the a size of SROWS by SCOLS. This simplifies addressing and 
pixel pointer to the next pixel to be processed and transfers therefore speed performance although two rows of the 
to strip complete check 1411. If an additional pixel or pixels 45 memory are not utilized. « 
remain to be processed in the strip, strip complete check Store vertical spatial gradient factor process 1423 trans- 
1411 transfers to process 1412 and processes 1412 to 1415 fers processing to update pixel pointer process 1424 that 
are repeated until all pixels within the RROWS rows (FIG. changes indices (i j) of the pixel pointer to the next pixel to 
13) wide and NCOLS columns long strip, i.e., the current be processed and transfers to strip complete check 1421. If 
strip, in in-strip memory 1104 are processed. When all the 50 an additional pixel or pixels remain to be processed, strip 
pixels in memory 1104 have been digitally noise reduced by complete check 1421 transfers to process 1422 and pro- 
unit 1110, the current strip of pixels has been replaced with cesses 1422 to 1424 are repeated until all pixels within the 
digitally noise reduced pixels and processing is transferred (SROWS-2) rows (FIG. 13) wide and SCOLS columns long 
to edge processor 1120. strip in in-strip memory 1104 are processed by edge pre- 

Within edge processor 1120, edge preprocessor 1121 55 processor 1121. When all these pixels in memory 1104 have 

generates a plurality of weighted three pixel horizontal and been processed, the weighted three-pixel vertical spatial 

vertical spatial gradient factors. These spatial gradient fac- gradient factor are all stored in vertical sum memory 1105 

tors are used repeatedly in processes described more com- and processing transfers to initialize pixel pointer process 

pletely below to reduce the complexity of post-processor 1425. 

1100 which in turn enhances the real-time performance of 60 Initialize pixel pointer process 1425 sets indices (i j) of 

post-processor 1100. the pixel pointer to location (0,1) in in-strip memory 1104, 

A process flow diagram for the processes performed by since the second column is the first column having a column 

edge pre-processor 1121 is presented in FIG. 14C In this of pixels on each side. Upon completion of initialize pixel 

embodiment, the vertical spatial gradient factors are gener- pointer process 1425, processing transfers to strip complete 

ated first and then the weighted horizontal spatial gradient 65 check 1426. 

factors are generated. Thus, initialize pixel pointer process Strip complete check 1426 determines whether any pixels 

1420 sets a pointer to a pixel in in-strip memory 1104 to the remain in in-strip memory 1104 that require processing by 
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edge pre-processor 1120 to generate a horizontal spatial 
gradient factor. In this embodiment, the last pixel processed 
is at location (SROWS, SCOLS-1) because column 
SCOLS-1 is the last column that has a column of pixels on 
each side. Thus, in this embodiment, edge pre-processor 5 
1121 processes a strip that is (SOWS) rows (FIG. 13) wide 
and (SCOLS-2) columns long to generate the weighted three 
pixel horizontal spatial gradient factors. If indices (i j) of the 
pixel pointer are less than (SROWS, SCOLS-1), processing 
transfers to generate horizontal spatial gradient factor pro- 10 
cess 1427 and otherwise to initialize pixel pointer process 
1430 (FIG. 14D) in edge detection unit 1122. 

Generate horizontal spatial gradient factor process 1427 
generates a weighted three pixel horizontal spatial gradient 
factor for the current pixel Q(ij). Specifically, in this 15 
embodiment, generate horizontal spatial gradient factor pro- 
cess 1427 generates a weighted horizontal spatial gradient 
factor HORSUM(i j) that is defined as: 

HORSUM(y>Q(y-l)+2G(y)+0(W+l) 2Q 

where Qn(ij) is a pixel in in-strip memory 1104. The 
relationship between the three pixels used to generate 
weighted horizontal spatial gradient factor HORSUM(ij) 
also is illustrated in FIG. 16. Specifically, pixel Qn(ij-l) is 
immediately to the left of current pixel Qn(i j) and pixel 25 
Q(ij+1) is immediately to the right of current pixel Q(ij). 

Upon completion of generate horizontal spatial gradient 
factor process 1427, processing transfers to store horizontal 
spatial gradient factor process 1428. Store horizontal spatial 
gradient factor process 1428 stores weighted horizontal 30 
spatial gradient factor HORSUM(ij) in horizontal sum 
memory 1106 at location (i,j). Notice that only SROWS by 
SCOLS-2 are used in horizontal sum memory 1106, which, 
as described above, has a size of SROWS by SCOLS. This 
also simplifies addressing and therefore speed performance 35 
although two columns of the memory are not utilized. 

Store horizontal spatial gradient factor process 1428 
transfers processing to update pixel pointer process 1429 
that changes indices (ij) of the pixel pointer to the next pixel 
to be processed in raster scan order and transfers to strip 40 
complete check 1426. If an additional pixel or pixels remain 
to be processed, strip complete check 1426 transfers to 
process 1427 and processes 1427 to 1429 are repeated until 
all pixels within SROWS rows (FIG. 13) wide and (SCOLS- 
2) columns long strip in in-strip memory 1104 are processed. 45 
When all these pixels in memory 1104 have been processed, 
the weighted horizontal spatial gradient factor are stored in 
horizontal sum memory 1106, and processing is transferred 
to edge detection unit 1122. 

FIG. 14D is a process flow diagram for the operations 50 
performed by edge detection unit 1122. As explained more 
completely below, edge detection unit 1122 utilizes differ- 
ential operators to obtain a plurality of spatial luminance 
gradients Gk(i j), where index k represents a particular edge 
orientation within a three pixels-by-three pixels window 55 
about current pixel Qn(ij), that is being processed. As 
explained above, these differential operators perform dis- 
crete differentiation on the pixels in the three pixels-bythree 
pixels window to measure the luminance change along 
different directions. 60 

Specifically, a plurality of differential operators Hk(i j) is 
based on Sobel differential operators. In this embodiment, 
only Sobel operators HI and H2 (See expression (3)) are 
used to generate horizontal spatial gradient Gl(ij) and 
vertical spatial gradient G2(ij) which in turn are used to 65 
generate a pixel gradient that is used in the edge identifica- 
tion process for pixel Qn(i,j). 
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Initially, in initialize pixel pointer process 1430, a pointer 
to a pixel in in-strip memory 1104 is set to the first pixel 
processed by edge detection unit 1122. In this embodiment, 
initialize pixel pointer process 1430 sets indices (i j) of the 
pixel pointer to location (1,1) in in-strip memory 1104, since 
this is the first pixel that is in a three pixel-by-three pixel 
window. Upon completion of initialize pixel pointer process 
1430, processing transfers to strip complete check 1431. 

Strip complete check 1431 determines whether any pixels 
remain in in-strip memory 1104 that require processing by 
edge detection unit 1122. In this embodiment, the last pixel 
processed is at location (SROWS-1, SCOLS-1) because 
again, this is the last pixel that is centered in a three 
pixel-by-three pixel window. Thus, in this embodiment, 
edge detection unit 1122 processes a strip that is (SOWS-2) 
rows (FIG. 13) wide and (SCOLS-2) columns long which is 
a RROWS by NCOLS strip with a one pixel border on all 
sides. If indices (ij) of the pixel pointer are less than 
(SROWS-1, SCOLS-1) processing transfers to generate 
spatial gradient process 1432 and otherwise to initialize 
pixel pointer process 1440 in switched filter unit 1131. 

In generate spatial gradient process 1432, for pixel Qn(i, 
j), weighted three pixel vertical spatial gradient factor 
VERSUM(ij-l) and VERSUM(ij+l), that are stored in 
vertical sum memory 1105, are retrieved. Similarly, 
weighted three pixel horizontal spatial gradient factor 
HORSUM(i-lj) and HORSUM(i+lj), that are stored in 
horizontal sum memory 1106, are retrieved. In this 
embodiment, horizontal spatial gradient Gl(ij), correspond- 
ing to a vertical edge through pixel Qn(ij), is defined as: 

Gl(y>VERSUM(y-l)-VERSUM(y+l) 

and vertical spatial gradients G2(ij), corresponding to a 
horizontal edge through pixel Qn(ij), is defined as: 

G2(y>HORSUM(i-l /)-HORSUM(i+l j) 

Thus, generate spatial gradient process 1432 generates the 
two spatial gradients using the retrieved spatial gradient 
factor and the above definitions. 

Spatial gradients Gl (ij) and G2 (ij) are identical to those 
defined above. However, in this embodiment the two spatial 
gradients are generated using the stored weighted three pixel 
spatial gradient factor. The use of the stored vertical and 
horizontal weighted three pixel spatial gradient factor 
reduces the number of operations required for generation of 
the spatial gradients by approximately a factor of two 
compared to the embodiments described above. 

Upon completion of generate spatial gradients process 
1432, edge detection unit 1122 transfers processing to gen- 
erate pixel gradient process 1433. In generate pixel gradient 
process 1433, pixel gradient A(ij) is defined as: 

*(4)MW|GlttJ)k|G2MD. 

where (Vs) is used for normalization. This definition is 
identical to that described above. Thus, the same process, as 
described above, is used to generate pixel gradient A (ij) in 
process 1433 and then processing transfers to generate edge 
threshold process 1434. 

In generate edge threshold process 1434, the two distinct 
thresholds used in the embodiments of the edge detection 
unit described above are replaced by a single threshold. In 
this embodiment, an adaptive edge threshold is defined as: 

THRSH^f/8)*(VERSUM(/J)+HORSUM(y) 

where eight is a normalization constant and t is a threshold 
adjustment factor that is taken as 0.20. This threshold is 
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modified intensity threshold that facilitates using stored embodiment, initialize pixel pointer process 1440 sets indi- 

weighted vertical spatial gradient factors and weighted hori- ces (i,j) of the pixel pointer to location (2,2) in in -strip 

zontal spatial gradient factors to reduce processing per- memory 1104, since this is the first pixel in the current strip 

formed by the post-processor. Thus, in generate edge thresh- that is not a border pixel. Upon completion of initialize pixel 

old process 1434, the appropriate three pixel weighted 5 pointer process 1440, processing transfers to strip complete 

spatial gradient factors are retrieved from memories 1105 check 1441. 

and 1106 and used to generate adaptive edge threshold Strip complete check 1441 determines whether any pixels 

THRSH, as defined above. If adaptive edge threshold remain in in-strip memory 1104 that require processing by 

THRSH is less than 80, adaptive edge threshold is set to 80 switched filter unit 1131. In this embodiment, the last pixel 

in this embodiment. A minimum threshold level is used to 10 processed is at location (SROWS-2, SCOLS-2) because 

eliminate spurious edges, which are areas where the lumi- again, this is the last pixel that is not a border pixel in the 

nance contrast is too low for the human eye to perceive an strip. Thus, in this embodiment, switched filter unit 1131 

edge. Upon generation of adaptive edge threshold THRSH, processes a strip that is (SOWS -4) rows (FIG. 13) wide and 

processing transfers from generate edge threshold process (SCOLS-4) columns long which is a RROWS by NCOLS 

1434 to edge identification process 1435. 15 strip. If indices (ij) of the pixel pointer are less than 

In edge identification process 1435, pixel gradient A(i j) is (SROWS-2, SCOLS-2) processing transfers from check 

compared with adaptive edge threshold THRSH. If pixel 1441 to edge pixel check 1442 and otherwise to initialize 

gradient A(ij) is greater than adaptive edge threshold pixel pointer process 1470 (FIG. 14G). 
THRSH, pixel Qn(ij) in in-strip memory 1104 is identified Edge pixel check 1442 determines whether current pixel 

as an edge pixel and conversely. Upon identification of pixel 20 Qn(i,j) was identified as an edge pixel by edge detection unit 

Qn(ij), processing transfers to edge pixel check 1436. 1122. Specifically, edge pixel check 1442 determines 

Edge pixel check 1436 transfers processing to set edge whether the edge flag at location (i,j) in edge-strip memory 

flag process 1437 if pixel Qn(i,j) was identified as an edge 1107 is set. 

pixel and to set not edge flag process 1439 otherwise. Set If the edge flag is set, current pixel Qn(i j) is an edge pixel, 

edge flag process 1437 sets an edge flag at a location (ij) in 25 and so no filtering is done. Thus, edge pixel check 1442 

edge-strip memory 1107 to indicate that pixel Qn(i,j) is an transfers to output pixel process 1449 (FIG. 14E) which in 

edge pixel. Conversely, set not edge flag process 1439 clears turn copies current pixel Qn(i,j) from in-strip memory 1104 

an edge flag at location (ij) in edge-strip memory 1107 to to output-strip memory 1109. Output pixel process 1449 

indicate that pixel Qn(ij) is not an edge pixel. Thus, the transfers processing to update pixel pointer process 1463 

location in edge -strip memory 1107 is set to indicate 30 (FIG. 14F) in quantizer scale adaptation unit 1132. 
whether pixel Qn(i,j) is an edge pixel. Consequently, when If the edge flag is not set, current pixel Qn(i,j) is not an 

all the pixels have been processed, edge-strip memory 1107 edge pixel, and so check 1442 transfers to copy pixel process 

contains an edge map for the pixels in the current strip. 1443. Copy pixel process 1443 copies current pixel Qn(ij) 

Both set edge flag process 1437 and set not edge flag to temporary pixel TEMP__PIXEL and transfers to classify 
process 1439 transfer processing to update pixel pointer 35 pixel process 1444. In classify pixel process 1444, current 
process 1438. Update pixel pointer process 1438 adjusts pixel Qn(i j) is processed to determine whether current pixel 
indices (ij) so that indices (ij) point to the next pixel in Qn(i j) is one of an edge boundary pixel and a shade pixel, 
raster scan order that is processed by edge detection unit As explained more completely below, the edge flag infor- 
1122. Update pixel pointer process 1438 transfers to strip mation in edge-strip memory 1107 for a three pixels-by- 
complete check 1431. 40 three pixels window about current pixel Qn(ij) is used by 

As explained above, strip complete check 1431 deter- classify pixel process 1444. After current pixel Qn(ij) is 

mines whether any pixels remain in in-strip memory 1104 classified in pixel classification unit 1444, processing trans- 

that require processing by edge detection unit 1122, In this fers to a filter unit 1150 within switched filter unit 1131. 
embodiment, the last pixel processed is at location In shade pixel check 1445, if current pixel Qn(ij) is a 

(SROWS-1, SCOLS-1) because again, this is the last pixel 45 shade pixel, processing transfers to two-dimensional filter 

that is centered in a three pixel-by-three pixel window. 1446, and otherwise to edge boundary pixel check 1447, In 

When the strip in in-strip memory 1104 has been processed two-dimensional filter 1446, temporary pixel TEMP_ 

by edge detection unit 1122, an edge map is stored for a PIXEL is replaced by the output pixel of a two-dimensional 

RROWS wide and NCOLS column long strip and a one filter with current pixel Qn(ij) as the input pixel, as 

pixel boundary around the strip in edge-strip memory 1107. 50 described more completely below. Two-dimensional filter 

The edge map, as described above, is a plurality of edge 1446, upon replacement of temporary pixel TEMP PIXEL, 

flags where for each pixel, the edge flag is set if the pixel is transfers processing to generate weight process 1460 in 

on an edge and is cleared if the pixel is not on an edge. In quantizer scale adaptation unit 1132. 
this embodiment, filter processor 1130 uses the edge map In edge boundary pixel check 1447, if current pixel 

without further processing, 55 Qn(i j) was classified an edge boundary pixel-by-pixel clas- 

Specifically, switched filter unit 1131 in filter processor sification unit 1451, processing transfers to one-dimensional 

1130 determines the filtering applied to each pixel by filter 1448, and otherwise to generate weight process 1460 

examining the edge map for pixels within a three pixels- in quantizer scale adaptation unit 1432. In one-dimensional 

by-three pixels window surrounding current pixel Qn(ij). As filter 1448, temporary pixel TEMP_PIXEL is replaced by 

explained above, pixel Qn(i j) is classified as either from an 60 the output pixel of an appropriate one-dimensional filter 

edge, edge boundary, or "shade" (smooth) area. Pixels along with current pixel Qn(i j) as the input pixel, as described 

edges are left unchanged. Pixels along edge boundaries are more completely below. 

directionally filtered with a one-dimensional filter, and shade In generate weight process 1460 (FIG. 14F), a weight 

pixels are low pass filtered with a two-dimensional filter. factor A is generated for use in the quantizer scale adaptation 

Specifically, in initialize pixel pointer process 1440 (FIG. 65 filtering. Recall, as described above, a quantizer scale was 

14E), a pointer to a pixel in in-strip memory 1104 is set to stored in quantizer scale memory 1108 for each macroblock 

the first pixel processed by switched filter unit 1131. In this in the current frame in current frame memory 1103. Thus, 
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generate weight process uses indices (STRIP_START+i, j) 
to define the macroblock containing current pixel Qn(ij) and 
retrieves the quantizer scale of the macroblock. In this 
embodiment, the quantizer scale is retrieved for each pixel. 
However, in another embodiment, the quantizer scale could 
be retrieved only once per macroblock. 

After the quantizer scale is retrieved, generate weight 
process 1460 accesses a weight look-up table stored in a 
memory of post-processor 1100 to determine weight A. 
Since the quantizer scale can only take one of a small 
number of finite values, only a small look-up table is needed 

Table 2 is one example of a look-up table for use with a 
H.261 decoder. For a H.261 encoder, the quantizer scale 
varies from 1 to 31 (quantization step size q equals 2*QS 



In initialize pixel pointer process 1470 (FIG. 14G), a 
pointer to a pixel in output-strip memory 1109 is set to 
location (2,2), since this is the first pixel in the current strip 
that is not a border pixel. Upon completion of initialize pixel 
5 pointer process 1470, processing transfers to copy strip 
process 1471. 

Copy strip process 1471 copies the rectangular strip of 
pixels defined by (2,2) <» (i j) < (SROWS-2, SCOLS-2) to 
output frame memory 1140. Specifically, out-strip pixel (ij) 
no is copied to location (STRIP_START+i-2, j-2) in output 
frame memory 1140. When the copy process is complete, 
processing transfers to update strip index process 1472. 

Update strip index process 1472 updates the value of 
initial row strip pointer STRIP_START so that the pointer 



and therefore varies from 2 to 62). Hence, using the look-up 35 points to the first row in the next strip to be processed. In this 



table, generate weight process 1160 obtains a weighting 
factor A for quantizer scale QS for current pixel Qn(i j). 
Processing transfers from generate weight process 1460 to 
retrieve pixel process 1461. 

TABLE 2 



Quantizer 


Weighting 


Scale (OS) 


Factor (A) 


1 


0.00 


2 


0.10 


. 3 


0.20 


4 


0.40 


5 


0.50 


6 


0.80 


QS ^ 7 


1.00 



embodiment, update strip index process 1472 redefines 
initial row strip pointer STRIP__START as: 

STRIP_START=STRIP_START+(SROWS-4). 

20 Update strip index process 1472 (FIG. 14G) transfers 
processing to frame complete check 1407 (FIG. 14A). 
Recall that in frame complete check 1407, post-processor 
1100 compares initial row strip pointer STRIP_START with 
number of rows NROWS in the frame. If initial row strip 
25 pointer STRIP__START is greater than or equal to number of 
rows NROWS, the post-processing of the frame is complete 
and so processing transfers to copy frame process 1473. 
Copy frame process 1473 copies the frame of post-processed 
pixel data from output frame memory 1140 to reference 
30 frame memory 1101 and transfers processing to update 
temporal index process 1402. 

If all the frames in the sequence have been post- 
processed, sequence complete check 1403 transfers to done 
process 1404 and post-processing is complete. Conversely, 



location (i,j) by forming a weighted combination of post- 
processed temporary pixel TEMP_PIXEL and current digi- 
tally noise reduced pixel Qn(i,j). Specifically, 

Outstrip Rxel(i, j) = A* 7£MP_PtXEL +(l-A)* Qn{U j) 



Upon generation of outstrip pixel (i j) process 1462 transfers 
to update pixel pointer process 1463. 



Retrieve pixel process 1461 retrieves current pixel Qn(ij) 
from in-strip memory 1104 and transfers processing to 
generate output pixel process 1462. Generate output pixel 

process 1462 creates a pixel in output-strip memory 1109 at 35 each frame is the sequence is processed as described above 

with respect to FIGS. 14A to 14G. 

Quantizer scale adaptive post-processor 1100 with quan- 
tizer scale adaptation was tested on a variety of test 
sequences. The test sequences included CIF as well as SIF 
40 sequences coded using a H.261 coder at rates ranging from 
low (96 kbps) to fairly high (384 kbps). In all cases, 
post-processor 1100 resulted in a significant improvement in 
the perceptual quality of the coded sequences while preserv- 
ing detail in areas coded with low quantizer scales. In 
Update pixel pointer process 1463 adjusts indices (i j) so 45 particular the quantizer scale adaptation ensured that there 
that indices (i,j) point to the next pixel in raster scan order was no undesirable softening in finely quantized areas, 
that is processed by switched filter unit 1131. Update pixel FIG. 16 is one embodiment of a more detailed process 
pointer process 1463 transfers to strip complete check 1441. diagram for filter unit 1150. As explained above, each pixel 
As explained above, strip complete check 1441 deter- in the current strip is individually processed in switched 
mines whether any pixels remain in in-strip memory 1104 50 filter unit 1131. 

that require processing by switched filter unit 1131. If As explained above, classify pixel process 1444 analyzes 
indices (ij) of the pixel pointer are less than (SROWS-2, the edge map for a three pixels-by-three pixels window 
SCOLS-2) processing transfers from check 1441 to edge about the current pixel in edge-strip memory 1107. A filter 
pixel check 1442 and processes 1442 to 1449 and 1460 to control flag is set equal to the number of pixels in the three 
1463 are performed as required. Thus, each pixel, that is not 55 pixels-by-three pixels window including the current pixel 
an edge pixel, is sequentially processed by switched filter 
unit 1131 and quantizer scale adaptation unit 1132 and the 
resulting pixel stored in output-strip memory 1109. Edge 
pixels are stored directly in output-strip memory 1109. 

When all the pixel data in in-strip memory 1104 is 60 
processed, output-strip memory 1109 contains edge pixels, 
that have not been filtered; and edge boundary pixels and 
shade pixels, that have been filtered by both switched filter 
unit 1131 and quantizer scale adaptation unit 1132. Thus, 

post-processing of the data in in-strip memory 1104 is 65 pixels window are not edge pixels implies that all pixels 
complete and so strip complete check 1441 transfers to processed by two-dimensional filter process 1446 are on the 
initialize pixel pointer process 1470. same side of an edge and do not span an edge. Therefore, 



that do not have the edge flag set. In this embodiment, the 
filter control flag ranges from one to nine. Upon completion 
of classify pixel process 1444, processing in switched filter 
unit 1131 transfers to shade pixel check 1445. 

Shade pixel check 1445 transfers processing to two- 
dimensional filter process 1446 if the filter control flag is 
nine, i.e., if none of the pixels in the three-by-tbree window 
have the edge flag set so that the current pixel is a shade 
pixel. Ensuring that all pixels in the three pixels-by-three 
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edges are not smeared by two-dimensional filter process non-edge pixels through the current pixel. Specifically, FIG. 
1446. If the filter control flag is less than nine, shade pixel 10A illustrates the current window configuration in the edge 
check 1446 transfers processing to horizontal edge check pixel map that is detected by vertical line of non-edge pixels 
1602, that is described more completely below. check 1603. If the pixel immediately above current pixel 

In this embodiment, two-dimensional filter process 1446 s 1001 and the pixel immediately below current pixel 1001 are 
utilizes a two-dimensional low pass filter. As explained non-edge pixels, vertical line of non-edge pixels check 1603 
above, two-dimensional filter process 1446 removes the is true and otherwise false. 

mosquito noise and the blocking artifacts in image areas at In FIGS. 10A to 10D, a non-edge pixel is represented by 
a distance greater than or equal to one pixel horizontally and a zero to indicate that the edge flag is not set for that pixel, 
vertically from edge pixels. The two-dimensional low pass aQ Pixels that are not on the vertical axis through the current 
filter is a separable three-tap low pass filter (LPF) in each pixel are represented by an "x", because the state of the edge 
direction with a mask of the form: flag for these pixels is a don't care state. If vertical line of 

non-edge pixels check 1603 is true, processing transfers to 
one-dimensional vertical filter process 1604 and otherwise 
to forty-five degree line of non-edge pixels check 1605. 
15 In one -dimensional vertical filter process 1604, the ver- 
tical line of non-edge pixels in the current window is filtered 
by a three-tap filter. In this embodiment, the three pixels are 
processed by a filter of the form \}/aM.,Va] and the filter 
output signal replaces temporary pixel TEMP_PIXEL. 
20 In this embodiment, for current pixel Qn(i,j), one- 
In post-processor 1100, the two dimensional low pass dimensional filter process 1604 retrieves the stored vertical 
filter is defined as: spatial gradient factor in vertical sum memory 1105 at 

location (ij) and then defines temporary pixel TEMP_ 
temp PIXEL = PIXEL as: 

25 

VERSUM (/, / - 1) + 2VERSUM (i, j) + VERSUM (i, j + 1) TEMP_PIXEL=VERSUM( y)/4. 

16 Upon completion of the filtering process, one -dimensional 
vertical filter process transfers 1604 to generate weight 

Thus, for current pixel Qn(i,j), the three weighted three pixel process 1640. 

vertical gradient factors defined above, are retrieved from 30 Forty-five degree line of non-edge pixels check 1605 

vertical sum memory 1105 and used to generate the filter determines whether the current three pixels-by-three pixels 

output signal from the two-dimensional filter. The use of the window includes a forty-five degree line of non-edge pixels 

stored information to implement the filter reduces the com- through current pixel 1001. Specifically, FIG. 10C illustrates 

plexity of filtering. The filter output signal replaces the value the current window configuration in the edge pixel map that 

of temporary pixel TEMP_PIXEL and processing transfers 35 * detected by forty-five degree line of non-edge pixels 

to generate weight process 1460. check 1605. If the pixel diagonally above and to the left of 

Recall that if the filter control flag is less than nine, shade current pixel 1001 and pixel diagonally below and to the 

pixel check 543 transfers processing to horizontal line of ri S ht of current P™ 1 1001 are non-edge pixels, forty-five 

non-edge pixels check 1601. Horizontal line of non-edge de 8 ree hne of non-edge pixels check 1605 is true and 

pixels check 1601 determines whether the current window 40 otherwise false. If forty-five degree line of non-edge pixels 

includes a horizontal line of non-edge pixels through the check 1605 fe true > processing transfers to one -dimensional 

current pixel. Specifically, FIG. 10B illustrates the current forty-five degree filter process 1606 and otherwise to one 

window configuration in the edge pixel map that is detected hundred thirty-five degree line of non-edge pixels check 

by horizontal line of non-edge pixels check 1601. If the pixel 1607. 

immediately to the left of current pixel 1001, the pixel 45 In one-dimensional forty-five degree filter process 1606, 
immediately to the right of current pixel 1001 are non-edge the forty-five degree line of non-edge pixels in the current 
pixels, horizontal line of non-edge pixels check is true and window are filtered by a three-tap filter. In this embodiment, 
otherwise false. If horizontal line of non-edge pixels check the three P ixels are processed by a filter of the form 
1601 is true, processing transfers to one-dimensional hori- and the filter 0Ut P ut replaces temporary pixel TEMP_ 
zontal filter process 1602 and otherwise to vertical line of 50 p f x EL. Upon completion of the filtering process, one- 
non-edge pixels check 1603. dimensional forty-five degree filter 1606 process transfers to 

In one-dimensional horizontal filter process 1602, the generate weight process 1460. 
horizontal line of non-edge pixels in the current window are 0ne hundred thirty-five degree line of non-edge pixels 
filtered by a three-tap filter. In this embodiment, the three check 1607 determines whether the current three pixels-by- 
pixels are processed by a filter of the form [W.Vi.W] and the 55 three P ixek window includes a one hundred thirty-five 
filter output signal replaces temporary pixel TEMP _PIXEL. dc 8 rec line of non-edge pixels through current pixel 1001. 

In this embodiment, for current pixel Qn(i,j), one- Specifically, FIG. 10D illustrates the current window con- 
dimensional filter process 1602 retrieves the stored spatial figuration in the edge pixel map that is detected by one 
gradient factor in horizontal sum memory 1106 at location hundred thirty-five degree line of non-edge pixels check 
(ij) and then defines temporary pixel TEMP_„PIXEL as: 60 1607 * If the P lxel diagonally below and to the left of current 

pixel 1001 and the pixel diagonally above and to the right of 

-reMP_PrXEL-HORSUMM/4. current pixd im m nQn . cdge Qne hundred thirly . 

Upon completion of the filtering process, one -dimensional five degree line of non-edge pixels check 1607 is true and 

horizontal filter process 1602 transfers to generate weight otherwise false. If one hundred thirty-five degree line of 

process 1460. 65 non-edge pixels check 1607 is true, processing transfers to 

Vertical line of non-edge pixels check 1603 determines one-dimensional one hundred thirty-five degree filter 1608 

whether the current window includes a vertical line of process and otherwise to generate weight process 1640. 
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In one-dimensional one hundred thirty-five degree filter Post-processor U00 processes CIF image 1701A, as 

process 1608, the one hundred thirty-five degree line of described above, and provides frame by frame, a 288 pixel 

non-edge pixels in the current window are filtered by a by 352 pixel frame to vertical interpolation unit 1702A. The 

three-tap filter. In this embodiment, the three pixels are quantizer scales provided by the decoder are used directly by 

processed by a filter of the form \}A,V2,Ya\ and the filter 5 post-processor 1100 because the quantizer scales and the 

output signal replaces temporary pixel TEMP_PIXEL. image being processed are in the same format. 

Thus, in this embodiment of switched filter unit 1131, Using techniques known to those skilled in the art, 

pixels directly adjacent to edges are not processed with the vertical interpolation unit 1702A converts the 288 pixel by 

two-dimensional filter because such a filter would include 352 pixel frame to a 480 pixel by 352 pixel frame that is 

pixels from either sides of an edge. However, it is desirable ^ provided to horizontal interpolation unit 1703A. Again, 

to clean up the area next to edges (edge border areas) to the using techniques known to those skilled in the art, horizontal 

maximum extent possible without smearing the edges since interpolation unit 1703A converts the 480 pixel by 352 pixel 

this results in clean sharp edges which are critical to the frame to a 480 pixel by 704 pixel frame that is provided to 

perceived quality of an image. Thus, the window is exam- display unit 1704A. 

ined to see if all the pixels lying along one of the four FIG. 17B illustrates another embodiment for translating a 

possible axes through the window are not edge pixels. If an 35 CIF image to a CCIR-601 image including quantizer scale 

axis is made-up of non-edge pixels, the pixels on that axis adaptive post-processor 1100. In this embodiment, the 

are processed with a one-dimensional filter. frames in the CIF image are not directly available from the 

Notice that the four possible axes are checked sequen- decoder. Rather, CIF image 1701B is first vertically inter- 

tially and the directional filtering is performed along the first polated by vertical interpolation unit 1702B. The available 

axis along which all the pixels are not-edge pixels. 20 output signals from vertical interpolation unit are a 240 

Axis examination is stopped after the- first axis along pixel -by-352 pixel field zero and a 240 pixel-by-352 pixel 

which filtering is allowed is found. Although adjacent pixels field one. 

are not examined for continuity of direction, axis examina- Hence, in this embodiment, a first quantizer scale adaptive 

tion always proceeds in the order shown above. This ensures post-processor 1100A receives field zero as an input frame 

that adjacent pixels are classified similarly if ambiguity in 25 and a second quantizer scale adaptive post-processor 1100B 

classification exists receives field one as an input frame. In this embodiment, 

This process of axis selection and resultant one- vertical interpolation unit 1702B re-samples input CIF 

dimensional directional filtering is equivalent to finding ^f^ 1 ™ 1 ^ ™ e pOsm ?Sn^ 0 . f P™ 1 *, 1 ? ™ ™i? 

• . nf fn t . ^ , c ■ „ % „ ■ , _ * 1701B and CCIR image 1704B is illustrated in FIG. 18A, 

pixels adjacent to the edges, i.e., nndmg edge border areas, - OD , 1Q „ 10 A -n * * *u 1 r 

, nii . • 1 • t . j . J 1 * . . 18B and 18C. FIG. 18 A illustrates the vertical position of 

and filtering pixels in the edge border areas along a direction 30 seven ^ {q qw { mm mQ ^ ap 

parallel to the edges. This technique also provides edge imag / pixels with referE f ncc nnmeral { on ^ left hand ^ 

enhancement. and CCIR image pixels in field zero on the right hand side 

Edge pixels and pixels directly adjacent to the edge pixels ^ reference number iL FIG 18c iii ustr ates CIF image 

that are not selected for one-dimensional directional filtering pixels witn rc f e r en ce numeral i on the left hand side and 

are not post-processed. Leaving these pixels unchanged 35 CCIR image pixels in field one on the right hand side with 

ensures that sharpness of edges in the decoded image is not reference number i2. Here, reference numerals i, il, and i2 

degraded by post-processing. This implies that the pixels indicate a scale unit. 

adjacent to edges which do not qualify for one-dimensional Successive pixels in the CIF image (FIG. 18A) are a unit 

directional filtering are also treated like edges. This is distance apart using scale i. On the same scale, successive 

equivalent to coalescing edge segments with a small dis- 40 pixels in each of the CCIR fields (FIGS. 18B and 18 C) are 

continuity (1 to 2 pixels) into continuous contours. (6/5)*i units apart with field one (FIG. 18C) offset from field 

Therefore, the pixel classification process in switched filter zero by a distance of (3/5)* i. A CCIR frame 1704B includes 

unit 1131 compensates for the lack of edge linking and both field zero and field one and thus contains pixels 

tracing and allows post-processor 1100 to effectively use an obtained by vertically re -sampling CIF image 1701 A every 

edge map equivalent to those generated by more complex 45 (3/5) *i units. 

edge detectors. In vertical interpolation unit 1702B, a CCIR pixel located 

While use of quantizer scale adaptive post-processor 1100 at a non-integer multiple of i is obtained by combining the 

utilizes quantizer scales, there is no issue with obtaining the two CIF pixels nearest the location of the CC1F pixel. The 

necessary quantizer scales when the decoded image and the relative weights given to the two CIF pixels to generate the 

display image have the same format. However, in some 50 corresponding CCIR pixel by vertical interpolation unit 

situations, the decoded image is converted to a different 1702B are inversely proportional to their distance from the 

format for display. For example, a CIF image (288 pixels by CCIR pixel. CCIR pixels at integer locations are obtained by 

352 pixels) may be converted to a CCIR-601 image (480 copying the CIF pixel at that location in vertical interpola- 

pixels by 704 pixels). Experiments have shown that post- tion unit 1703B. 

processor 1100 enhances the image quality if post-processor 55 As explained above, post-processors 1100A and 1100B 

1100 is utilized directly on the CIF image, or alternatively, use the macroblock quantizer scales for varying the extent of 
is utilized between the vertical interpolation and the hori- postprocessing. The post -processed pixel is a weighted sum 
zontal interpolation from the CIF image to the CCIR-601 of the filtered pixel and digitally noise reduced pixel. The 
image. weight given to the filtered pixel increases with an increase 

FIG. 17A illustrates a first embodiment for translating a 60 in the quantizer scale (quantizer step-size) for the macrob- 

CIF image to a CCIR-601 image including quantizer scale lock to which the pixel belongs. 

adaptive post-processor 1100. CIF image 1701A from the In this embodiment, the quantizer scale used to obtain the 

decoder is 288 pixels by 352 pixels. In this embodiment, weighting factor in generate weight process 1460 (FIG. 14F) 

current frame memory 1103 and reference frame memory for a pixel was the quantizer scale for the pixel in CIF image 

1101 have 292 pixel by 356 pixel storage locations, while 65 1701B closest to that pixel. FIG. 18B illustrates the location 
input frame memory and output frame memory are 288 of the il th pixel in field zero on scale i. As shown in FIG. 
pixels by 352 pixels. 18B: 
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i-(6/5)xii Also, the edge maps from the two methods were generated 

and displayed on Sun workstations for each frame. The 
wnere video conferencing sequence was coded at 192 kbps at 15 

i is the vertical position of the pixel in the CIF frame; and frames per second, and at 128 kbps at 15 frames per second. 

11 is the vertical position of the corresponding pixel in 5 The edge-maps as well as the final CCIR sequences were 
vertically interpolated frame zero. indistinguishable from each other for the two methods 

Therefore, the vertical location of the closest CIF pixel to the demonstrated in FIGS. 17A and 17B. Therefore, post- 

11 th pixel being processed by post-processor 1100A is: P ro f ss ? r 1100 of , thls can be a PP lied to individual 

fields alter vertical interpolation without compromising per- 
Round ((6/5)x/3) 10 formance. 

In the above embodiment of quantizer scale adaptive 
which can be rewritten as: post-processor 1100, a particular combination of digital 

noise reduction and artifact reduction was used. However, 
i=(i2xii+5)/io the principles of this invention are applicable to any post- 

processor. For example, as illustrated in FIG. 19, input frame 
Thus, weight process 1460 in post-processor 1100A uses the 35 Fn from the decoder is stored in an input frame memory 
above expression to convert the vertical position of the 1901. 

current pixel to a vertical position in the CIF image so that Post-processor 1900 generates a post-processed frame 
the appropriate quantizer scale is retrieved. Qi- The particular operations in post-processor 1900 are not 

Similarly, in generate weight process 1460 in post- of particular importance. Post-processor 1900 could perform 
processor 1000B for field one, vertical position i of the CIF 20 digital noise reduction only, artifact reduction only, a corn- 
pixel corresponding to the pixel being processed is: bination of digital noise reduction and artifact reduction, or 

any other type of post-processing of interest to the user. For 
i'«3/5+(6/5)x*2 example, post-processor 1900 could be post-processor 300, 

as described above. 

wnere 25 Post-processed frame Qn is stored in a post-processed 

i is the vertical position of the pixel in the CIF frame; and frame memory 1956 in quantizer scale adaptation unit 1950. 

12 is the vertical position of the corresponding pixel in Decoded frame Fn is stored in a decoded frame memory 
vertically interpolated frame one. 1955. The macroblock quantizer scales for decoded frame 

Therefore, the vertical location of the closest CIF pixel to the Fn are stored in quantizer scale memory 1953, as described 

12 th pixel being processed by post-processor 1100B is: 30 above. A look up table for converting quantizer scales to 

weight factors is stored in look-up table memory 1954. 
Round (#5+(5/5)xj2) Filter unit 1951 in quantizer scale adaptation unit 1950, 

retrieves a decoded pixel Fn(ij) from memory 1956, and a 
which can be rewritten as: post-processed pixel (ij) from memory 1956. Filter unit 

»-fi2xi2+ 35 1951 generates a weight factor Aused in the quantizer scale 

'"^ Xi + ) adaptation filtering. 

Thus, weight process 1460 in post-processor 1100B uses the Filter unit 1951 ^ mdicc& fti) to definc the macroblock 

above expression to convert the vertical position of the containing current pixel Qn(i,j) and retrieves the quantizer 

current pixel to a vertical position in the CIF image so that sca A le r of * e macroblo( *. 

the appropriate quantizer scale is retrieved. In the above 40 After the quantizer scale is retrieved, filter unit 1951 uses 

expressions, the operator stands for integer division the WCL ^ X look - u P table stored ™ memory 1954 to deter- 

which truncates any fractional parts. mme wei S ht A * ^ ^-up table is the same as that 

As an example of the operations performed by generate dcsc ' lbed above. Filter unit 1951 generates a weighted 

weight process 1460 in post-processor 1100B, assume that combination of post-processed pixel Qn(i j) and decoded 

process 1460 must determine the closet pixel to the fourth 45 P lxel Fn ^' s P eci fi c aUy> 
pixel in field one, i.e., i2 equals four. The closest CIF pixel 

is: Output Pixel(i, j) = A* Qn{i, j) + (i - A) * Fn(r\ j) 



*-(12*4+ll)/10«59/10-5th pixel. 

Note that since the location of this pixel in i units is 27/5, 
the correct CIF pixel is identified and the correct quantizer 
scale is retrieved by this process. 

Thus, in this embodiment, quantizer scale adaptation unit 
1131 includes an extra process within generate weight 
process 1460 that translates the interpolated location of the 
current pixel to the location of the closest pixel in the 
original frame and retrieves the quantizer scale for the 
closest pixel in the original frame in determining the weight 
factor. 

To test the two embodiments illustrated in FIGS. 17Aand 
17B, 147 frames of a video-conferencing CIF sequence were 
used. The CIF sequence was coded at a variety of bit rates 
and frame rates using a H.261 encoder. A side-by side 
comparison of the resultant CCIR images was generated and 
played back in real-time on an Abekas image sequencer. 



50 In this embodiment output pixel(ij) is stored in output 
memory 1952. Thus, quantizer scale adaptation unit 1950 
filters each pixel based upon the pixels quantization. 
Specifically, this embodiment of a quantizer scale adaptive 
post-processor adapts to the coding parameters available at 
55 the decoder, e.g., a quantizer scale is used. Since the 
quantizer scale is indicative of the degree of quantization, 
the quantizer scale also is indicative of the amount of 
post-processing that is needed and the quantizer scale post- 
processing is independent of the particular type of post- 
60 processing used in post-processor 1900. Alternatively, as 
shown above, quantizer scale adaptation unit 1950 can be 
incorporated into post-processor 1900 to reduce memory 
requirements and to enhance the overall performance of the 
post-processing. 
65 In view of this disclosure the various units described 
herein can be implemented in either software or hardware or 
a combination of the two to achieve a desired performance 
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level. Therefore, the embodiments described herein are 7. A transform artifact reduction method for decoded 

illustrative of the principles of this invention and are not video pixel data as in claim 6 wherein said comparing a pixel 

intended to limit the invention to the specific embodiments gradient for a pixel further comprises: 

described. In view of this disclosure, those skilled in the art comparing said pixel gradient with an adaptive edge 

can implement the combination of a digital noise reduction 5 threshold. 

unit and an artifact reduction unit in a wide variety of ways. 8. A transform artifact reduction method for decoded 

Further, the artifact reduction unit can be used independent video pixel data as in claim 7 wherein said setting said edge 

of digital noise reduction, and both can be used indepen- flag for said pixel in said edge map upon said pixel gradient 

dently or in conjunction with quantizer scale adaptive fil- bein S greater than said threshold further comprises: 

tering, 10 setting said edge flag only upon said pixel gradient being 

We claim: greater than said adaptive edge threshold. 

1. A transform artifact reduction method for decoded 9 - A transform artifact reduction method for decoded 
video pixel data, said method comprising: video P ixel data as ^ claim 7 furlher comprising prior to said 

filtering selected pixels in a block of pixels in said comparing a pixel gradient with an adaptive edge threshold: 

decoded video pixel data using a spatially-variant filter; 35 generating said pixel gradient for said pixel, 

filtering said selected pixels from said spatially-variant J 0 * A ^sform artifact reduction method for decoded 

filter using a coding parameter adaptive filter, Vld ^ dala as m c ! aim 9 wherein generating said pixel 

, ... . * ** c *j 1.1 1 • u .l. gradient further comprises: 

wherein said characteristic of said macroblock is whether & . \. . . 

said macroblock is a replenished macroblock. generating a plurality of spatial gradients for said pixel 

2. A transform artifact reduction method for decoded 20 *^g P^ls in a window about sa!d pixel. 

video pixel data as in claim 1 wherein said coding parameter . j 1 . ^nsform artifact reduction method for decoded 

adaptive filter processes a macroblock of said decoded video v L ldeo P 1 ^ 1 data as in claim 10 wherem said imadaw 15 a 

pixel data using a coding parameter for said macroblock. . three pixels-by-three pixels wmdow with said pixel centered 

3. A transform artifact reduction method for decoded m ? aid three Pjxels-by-thrcc pixels window. 

video pixel data, said method comprising: 25 12- A transform artifact reduction method for decoded 

cu . 1 j 1 • 1 * ui 1 f • 1 • *j video pixel data as m claim 10 wherein said plurality of 

filtering selected pixels in a block or pixels in said 1 j- * • . 1 j- r 

* jt j • 1 • ij* • *• 11 • spatial gradients comprises two spatial gradients, 

decoded video pixel data using a spatially -variant filter: r . j „ ^ 

C1 . . , * n . 13. A transform artifact reduction method for decoded 

filtering said selected pixels from said spatially-variant vide0 ixd data M ^ claim 10 tin said ^ 

fi ter using a coding parameter adaptive filter, the dient ^ 

filtering said selected pixels from said spatially-variant ... ... c . t , , . 

filter using a coding parameter adaptive filter including combining said plurality of spatial gradients to obtain said 

oixel gradient 

filtering said selected pixels from said spatially -variant ia a * f 



filter using a quantizer scale adaptive filter 



14. A transform artifact reduction method for decoded 

, . , . . , . _ video pixel data as in claim 7 further comprising prior to said 

adapting said quantizer scale adaptive filter for a mac- 3J com p ari „g a pixel gradient with an adaptive edge threshold: 

roblock using a characteristic of said macroblock, said j 1 . , 

characteristic of said macroblock is whether said mac f™*™% *** adaptive edge threshold 

roblock is a replenished macroblock. .}*■ A tra f f orm . artl , fact ' ed " ctl0 . D me ! h ° d for 

4. A transform artifact reduction method for decoded v ! deo P ix , el dala a f »? claim ? ^rein said plurality of edge 

video pixel data as in claim 3 wherein said quantizer scale classifications includes an edge classification, and said pixel 

, 4 . ,,, U1 , r .,j j , 40 is assigned the edge classification if said edge flag for said 

adaptive filter processes a macroblock of said decoded video , r ., , e ... . • 

• i j 4 • i c j ui 1 Pixel m said edge map indicates that said pixel is an edge 
pixel data using a quantizer scale tor said macroblock. . - ° r r to 

5. A transform artifact reduction method for decoded A , e . e . , ■ ■ « <• 

• i • i j * • j *i. j. ■ • 16. A transform artifact reduction method for decoded 
video pixel data, said method comprising: ., • , , ■ i • - , • . i . <• * 

■ l4 . , , j . . i_i i r ■ i * -i video pixel data as in claim 5 wherein said plurality of edge 

filtering selected pixefe in a block of pixels in said 4J classifications includes an ed border cla ^ ificati ; n . 

decoded video pixel data usmg a spaUally-vanant filter; J? A transform ^ ct reduction method for decoded 

an video pixel data as in claim 16 wherein said pixel is assigned 

filtering said selected pixels from said spatially-variant said edge border classification if (i) the edge flag for at least 

filter using a coding parameter adaptive filter; one pixel in a of pixels about the pixel ^ not set; 

wherein filtering selected pixels in a block of pixels in 50 the edge flag for at least three pixels in a window of pixels 

said decoded video pixel data using a spatially-variant abaut the pixel are not set; and (iii) the pixel is in a line of 

filter further comprises: pixels in the window and the edge flag for each pixel in the 

generating an edge map for said block of pixels ii ne jg not set> 

wherein in said edge map, an edge flag for each pixel 18. A transform artifact reduction method for decoded 

in said block is configured to identify each pixel as 55 video pixel data as in claim 17 wherein said window is a 

one of an edge pixel and a non-edge pixel; and three pixels-by-three pixels window with said pixel centered 

assigning a pixel in said block of pixels one classifi- i n sa i d three pixels-by-three pixels window, 

cation in a plurality of edge classifications using 19. A transform artifact reduction method for decoded 

edge flags in said edge map for pixels in a window v i deo p i xe i data as in claim 17 wherein a pixel assigned said 

about said pixel. 60 ed g e border classification is filtered in a one-dimensional 

6. A transform artifact reduction method for decoded filter that processes said line of pixels. 

video pixel data as in claim 5 wherein said generating an 2 0. A transform artifact reduction method for decoded 

edge map further comprises: v i deo p i xeJ data as in claim ^ where said one-dimensional 

comparing a pixel gradient for a pixel with a threshold; filter is a low pass filter. 

anc * 65 21. A transform artifact reduction method for decoded 

setting said edge flag for said pixel in said edge map upon video pixel data as in claim 5 wherein said plurality of edge 

said pixel gradient being greater than said threshold. classifications includes a shade classification. 
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22. A transform artifact reduction method for decoded 
video pixel data as in claim 21 wherein said pixel is assigned 
said shade classification if the edge flag for said pixel and 
each pixel in a window of pixels about said pixel is not set. 

23. A transform artifact reduction method for decoded s 
video pixel data as in claim 22 wherein said window is a 
three pixels-by-three pixels window with said pixel centered 

in said three pixels-by-three pixels window. 

24. A transform artifact reduction method for decoded 
video pixel data as in claim 22 wherein a pixel assigned said 10 
shade classification is filtered in a two-dimensional filter that 
processes said window of pixels. 

25. A transform artifact reduction method for decoded 
video pixel data as in claim 24 where said two-dimensional 
filter is a low pass filter. 15 

26. A transform artifact reduction method for decoded 
video pixel data, said method comprising: 

digitally noise reducing a block of pixels to obtain a 

digitally noise reduced block of pixels; 
filtering selected pixels in said digitally noise reduced 20 

block of pixels using a spatially-variant filter; and 
filtering said selected pixels from said spatially-variant 

filter using a coding parameter adaptive filter. 

27. A transform artifact reduction method for decoded 25 
video pixel data as in claim 26 wherein said coding param- 
eter adaptive filter processes a macroblock of said decoded 
video pixel data using a coding parameter for said macrob- 
lock. 

28. A transform artifact reduction method for decoded 3Q 
video pixel data as in claim 26 further comprising: 

adapting said coding parameter adaptive filter for a mac- 
roblock using a characteristic of said macroblock. 

29. A transform artifact reduction method for decoded 
video pixel data as in claim 28 wherein said characteristic of 35 
said macroblock is whether said macroblock is a replenished 
macroblock. 

30. A transform artifact reduction method for decoded 
video pixel data as in claim 26 wherein filtering said selected 
pixels from said spatially-variant filter using a coding 40 
parameter adaptive filter comprises: 

filtering said selected pixels from said spatially-variant 
filter using a quantizer scale adaptive filter. 

31. A transform artifact reducdeco method for decoded 
video pixel data as in claim 30 wherein said quantizer scale 45 
adaptive filter processes a macroblock of said decoded video 
pixel data using a quantizer scale for said macroblock. 
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32. A transform artifact reduction method for decoded 
video pixel data as in claim 30 further comprising: 

adapting said quantizer scale adaptive filter for a mac- 
roblock using a characteristic of said macroblock. 

33. A transform artifact reduction method for decoded 
video pixel data as in claim 32 wherein said characteristic of 
said macroblock is whether said macroblock is a replenished 
macroblock. 

34. A transform artifact reduction method for decoded 
video pixel data, said method comprising: 

digitally noise reducing a block of pixels in said decoded 
video pixel data to obtain a digitally noise reduced 
block of pixels; and 
filtering selected pixels in said digitally noise reduced 
block of pixels using a coding parameter adaptive filter, 
said filtering said selected pixels in said digitally noise 
reduced block of pixels using a quantizer scale adaptive 
filter. 

35. A transform artifact reduction method for decoded 
video pixel data as in claim 34 wherein said coding param- 
eter adaptive filter processes a macroblock of said decoded 
video pixel data using a coding parameter for said macrob- 
lock. 

36. A transform artifact reduction method for decoded 
video pixel data as in claim 34 further comprising: 

adapting said coding parameter adaptive filter for a mac- 
roblock using a characteristic of said macroblock. 

37. A transform artifact reduction method for decoded 
video pixel data as in claim 36 wherein said characteristic of 
said macroblock is whether said macroblock is a replenished 
macroblock. 

38. A transform artifact reduction method for decoded 
video pixel data as in claim 34 wherein said quantizer scale 
adaptive filter processes a macroblock of said decoded video 
pixel data using a quantizer scale for said macroblock. 

39. A transform artifact reduction method for decoded 
video pixel data as in claim 34 further comprising: 

adapting said quantizer scale adaptive filter for a mac- 
roblock using a characteristic of said macroblock. 

40. A transform artifact reduction method for decoded 
video pixel data as in claim 39 wherein said characteristic of 
said macroblock is whether said macroblock is a replenished 
macroblock. 
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